Repository navigation
Conversation
LocalConversation.condense() used to process the condensation request by calling agent.step(), which also executes pending actions, including tool calls still waiting for user confirmation. Invoke the condenser directly instead so an explicit compaction only compacts history. The condense endpoint now maps the ValueError raised when no compatible condenser exists to HTTP 409 instead of a 500. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ersations
Add a conversation-scoped voice API under /api/conversations/{id}/voice,
restricted to conversations tagged as Insider controllers:
- GET reports availability, provider, model, and whether a run is active.
- POST /realtime brokers a WebRTC offer. On the default OpenAI provider the
server builds the Realtime session (instructions, bounded saved context,
send_to_insider and end_voice_call tools) and exchanges the SDP through
OpenAI's calls endpoint with the server-held API key.
- GET and DELETE /realtime/{call_id} poll and end a call, with call IDs
bound to their conversation.
Add an opt-in Codex Voice relay (voice_provider = "codex") that delegates
the call to a Codex app-server sign-in kept in a dedicated home directory,
configured through codex_voice_home.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Contributor
REST API breakage checks (OpenAPI) — ✅ PASSEDResult: ✅ PASSED |
1 task done
Build lifecycle test state with the Codex Voice manager created by create_app and verify it is awaited during shutdown. Co-authored-by: openhands <openhands@all-hands.dev>
Contributor
|
Traceability and validation update (2026-09-19): the original tested SDK worktree is now preserved at enyst/agent-sdk@7c806a919, on The six failing lifecycle cases were missing the Voice manager in their fake application state. The test-only follow-up adds that manager and verifies shutdown awaits it. Production behavior is unchanged. The corresponding fix is also preserved on the original tested branch at d5dc75ee6. Validation:
The HTTP regression verifies authentication, missing-provider availability, and safe rejection of condensation without a condenser. Pending-approval preservation is covered by the separate condensation unit test. Full CI still needs to complete on the new head; this update does not claim that the earlier |
Contributor
Coverage Report •
|
|||||||||||||||||||||||||||||||||||||||||||||
Validate and deduplicate realtime handoffs before dispatch, speak only saved results, and expose safe error codes for connection and request failures. Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
Contributor
|
📁 PR Artifacts Notice This PR contains a |
20 tasks
Expose typed voice setup, status and hangup through ConversationManager and the secondary clients entrypoint. Keep the generated schema pinned to its release and track these prerelease routes in the endpoint audit. Co-authored-by: openhands <openhands@all-hands.dev>
Document the additive API, saved-controller handoff path, safe compaction, typed client and fresh test evidence with pinned source links. Co-authored-by: openhands <openhands@all-hands.dev>
Preserve pinned-file links when HTMLPreview rewrites fragments, and contain figures inside the narrow document column. Co-authored-by: openhands <openhands@all-hands.dev>
31 tasks
Keep the generated contract aligned with the canonical 1.53.0 release while preserving the tracked prerelease voice endpoints. Co-authored-by: openhands <openhands@all-hands.dev>
Keep the Voice manager lifecycle checks alongside main's short tmux temporary-directory fixture. Co-authored-by: openhands <openhands@all-hands.dev>
Contributor
|
Progress update (9 Oct): I merged current main into this branch at b303764 and resolved the test-fixture conflict. The branch now merges cleanly. Locally, 139 focused Python tests, 122 selected TypeScript client tests, the TypeScript build and lint, and Ruff/pycodestyle passed. The new CI run reports nine failed image/integration jobs. The inspected jobs hit Docker Hub HTTP 429 while pulling This draft still needs real provider speech, device audio, and call cleanup tests with Canvas #17558 and Insider app #2. |
Keep the current server lifecycle and client release contract. Give the new tmux lifespan test fixtures the existing Voice manager dependency. Co-authored-by: openhands <openhands@all-hands.dev>
Preserve both Voice HTTP coverage and upstream configuration-isolation checks. Validated 170 focused Python tests, 397 TypeScript tests, hooks, type checks, and API schema compatibility. Co-authored-by: openhands <openhands@all-hands.dev>
Retain upstream typed condensation-unavailable handling while preserving condenser-only execution and the Insider Voice API. Co-authored-by: openhands <openhands@all-hands.dev>
Document conversation-bound Voice, delegation, cleanup and API failure recipes. Assert explicit condensation does not advance the agent. Co-authored-by: openhands <openhands@all-hands.dev>
Co-authored-by: openhands <openhands@all-hands.dev>
Codex V3 may keep a user transcript open across a delegated request and a spoken correction. Count each nonempty input delta so the correction suppresses the obsolete spoken answer while its saved task still finishes. Cover both a new transcript and a correction within the already open transcript through the real child-process protocol fixture.
Do not describe a spoken correction as forwarded, queued, or applied before the host confirms acceptance. Explain that a running saved task is not steered by later speech and an unsent request must be repeated afterward.
enyst
added
the
integration-test
Co-authored-by: openhands <openhands@all-hands.dev>
Preserve bounded provider, user-flow and media evidence, the historical failed replays, and explicit remaining device and workflow limits. Co-authored-by: openhands <openhands@all-hands.dev>
Contributor
|
Hi! I started running the integration tests on your PR. You will receive a comment with the results shortly. |
Contributor
🧪 Integration Tests ResultsOverall Success Rate: 95.8% 📁 Detailed Logs & ArtifactsClick the links below to access detailed agent/LLM logs showing the complete reasoning process for each model. On the GitHub Actions page, scroll down to the 'Artifacts' section to download the logs.
📊 Summary
📋 Detailed Resultslitellm_proxy_openai_gpt_5.5
litellm_proxy_deepseek_deepseek_v4.1_flash
Skipped Tests:
Failed Tests:
litellm_proxy_gemini_3.1_pro_preview
litellm_proxy_anthropic_claude_sonnet_4_6
Failed Tests:
litellm_proxy_minimax_MiniMax_M2.7
Skipped Tests:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HUMAN:
Experimental branches. Testing live voice with an Insider conversation that knows your other conversations and helps you keep on track.
This is a bit of a combination between the way of working with a personal agent, and the way of working with a visual web interface for conversations: one of the conversations helps you more keep track of the others.
AGENT:
Why
Realtime voice should reach the selected saved Insider Cat, execute work there, and speak its persisted result. An intermediate relay turn could previously omit the forwarding step.
Summary
Add availability, call setup, status and teardown APIs plus the TypeScript
VoiceClientandConversationManager.voice. Codex handoffs go directly to the saved conversation, with replay protection and fixed error categories distinguishing unsent work, uncertain results and transport failure. Explicit condensation returns 409 without a compatible condenser and does not advance pending approvals.Source: voice_router.py · codex_voice.py · voice-client.ts · local_conversation.py.
REST API contract changes
Compared with base OpenAPI
60192ed54db7for public/api/**paths.Issue Number
Refs #5515, which contains the shared acceptance criteria and validation status.
Related PRs
Related implementation: Canvas #17558 and Insider #2. Canvas issue #17557 tracks host behavior; notebook #24 records the current architecture and maintenance evidence.
How to Test
Integrated verification — 26 October 2026
The final integrated ledger has 42 passing and four unrun latest entry checks, retaining earlier failures and revision provenance; these are entry checks, not full feature-family coverage. See the dated verification record.
verify-agent-servernow documents F35 Voice and strengthens F10's no-extra-turn compaction assertions. The full map validates 35 families / 951 IDs; nine credential-free F35 API/client recipes passed on a fresh server. Browser/protocol requirements were explicitly blocked in that API-only replay rather than counted as passes.The paired UI run used a normal OpenHands Cat with actual tools and skills through the authorized Luna profile. A spoken backend-count request reached that Cat, performed the query and returned its saved result through real Codex Voice. A new call recalled saved context after
/condense; independent API checks found only the condensation request, condensation result and state update, with all 44 earlier event hashes unchanged. Generated microphone input and media statistics establish the tested provider/transport path, not physical-device audio. A final uninterrupted call after both fixes saved the spoken request, calculated2+3in the terminal, saved5and returned5through Voice; real response audio and clean End were measured.Pending-answer fencing in
29a83a65/27080d11, paired with App PR #4, still failed the first live spoken-correction retest. The additional open-transcript fence in2795340apassed 63 focused tests, a live-HTTP smoke check and a bounded real-provider replay: the old result remained saved but was not spoken after the correction, including more than 60 seconds after completion. The replay exposed a misleading acknowledgment of an unsent correction. SDKb09624ddpassed four protocol/busy-handoff tests and a subsequent live replay: Voice truthfully said the new request had not been sent and should be repeated after the saved task finished; the old saved result was not spoken. No complete raw-audio barge-in or pending-request steering claim is made.Main refresh — 23 October 2026
Head
c99daf37f8221f741b3364817e8678378931166emergesmainat60192ed54db73ac658ac1a07462916126c32d3a0(release 1.58.1). The single conflict was the condensation error response: the merged branch uses upstream's typedCondensationUnavailablehandling for HTTP 409 and retains the Insider change that invokes only the condenser, without executing pending tools. The four Voice endpoints and the App's profile-launch, appended context, confirmation, and active-profile contracts remain compatible.This is source and regression-test evidence. Existing services were not restarted; no fresh real-provider speech or physical iPad microphone/speaker check was performed. This PR remains draft, and current-head CI is running after the push.
Previously recorded verification
Run the focused Python suite from the repository root:
From
clients/typescript, run the client tests, build, lint, type-budget and API checks described in its README. The Voice tests cover authenticated requests, encoded identifiers, SDP payloads, fixed failures without retries, and cancellable polling.Merge update — 19 October 2026
Head
8a95ad604619465855cc608ed61cbb560778d8c4mergesmainat6ae64079cfd30cfe2f01a060c59168a4d650d883. The single conflict was adjacent imports in the real-HTTP regression file; the resolution keeps Voice coverage and upstream configuration-isolation checks. Voice endpoints, profile launch additions, confirmation settings, and active-profile response fields remain compatible with the Insider App. Upstream lifecycle, session-key export, and run-admission fixes are retained.Current-head CI will run after this push. No live provider call, physical iPad audio check, or existing service restart was performed in this maintenance pass; historical speech results below remain historical. The PR stays draft.
Merge update — 9 October 2026
The 9 October head
b30376433d9c8a7b6bc9bd59ea062308fddf7d15mergesmainat1cfba2117e92730b0fe1c32520a39aad8611bb8c. The sole text conflict was a lifecycle test; its resolution keeps the Voice manager teardown checks and the newer short tmux path fixture. All 139 focused Python cases passed across local runs, along with the TypeScript build, lint and format checks, and 122 selected client tests. The first CI attempt for this head hit Docker Hub HTTP 429 while fetchingpython:3.13-bookwormin Agent Server image builds and integration tests, so a complete green CI run is still pending. The provider and device limits described below still apply.Validation — 5 October 2026 (earlier head)
Earlier source tested at
2b602507af962badfe82027f2a14911dfc9e90e5, including a clean merge of then-currentmain(54daf056bd863bb46f922a2fe9324dd736b37ff6).The original macOS CI failure occurred during a PyPI download of
nodejs-wheel-binaries==22.19.0, before the build or tests started. The failed job passed on rerun; no production workaround was added for that network timeout.A later schema check ran while the Agent Server 1.53.0 release artifact was still being published. Once the canonical artifact was public, the targeted rerun passed without changing the schema or weakening the check.
Main refresh — 15 October 2026 (earlier head)
The original description also recorded integration head
339d850809c06bfe781c362c01375c8e081eeaaf, merging mainf5f849f5a: 165 focused Python cases and 386 TypeScript tests passed, alongside lint/build/formatting/type-budget/OpenAPI/endpoint checks and commit hooks. On macOS,TMPDIR=/tmpavoided tmux Unix-socket path-length failures. The separately installed Insider App had 129 passing tests and a reproducible committed browser bundle. No fresh provider-audio or physical-device check was performed.Video/Screenshots
The paired Canvas/App browser check used real Agent Server persistence and terminal execution with scripted model responses. The evidence notes identify the exact source revisions and verify saved Cat identity, New Cat/resume, compaction, and missing-key guidance; this does not claim live-provider speech.
Design Doc
Before/after design and validation page, with source links and API/lifecycle diagrams. This is a temporary
.pr/review artifact; the pinned link survives cleanup.Type
Notes
This remains an experimental draft. OpenAI is the default provider with client-side delegation; Codex requires explicit opt-in and a dedicated sign-in, with no automatic fallback. Status polling is for Codex calls; OpenAI state comes from WebRTC. Ending a call does not cancel accepted work. September 19–20 speech trials remain historical. The October 26 generated-input run supplies fresh real-provider and browser-media evidence, including a bounded stale-answer replay that passed after a further transcript fence. The subsequent acknowledgment-honesty repair also passed a live retest. Physical iPad microphone/speaker behavior remains unverified. The generated TypeScript schema stays pinned to Agent Server 1.58.1; the four Voice endpoints require this unreleased server change. The linked design page supplies the larger lifecycle diagrams, so the PR can keep the ownership sketch inline.
Visual explanation by Codex using HumanLayer show-me.
REST API contract changes
Compared with base OpenAPI
6ae64079cfd3for public/api/**paths.REST API contract changes
Compared with base OpenAPI
54daf056bd86for public/api/**paths.🐳 Agent Server images for this PR — GHCR package, pull/run commands, and all pushed tags (click to expand)
• GHCR package: https://github.com/OpenHands/agent-sdk/pkgs/container/agent-server
Variants & Base Images
eclipse-temurin:17-jdkpython-node-runtimepython-node-runtimepython-node-runtimegolang:1.21-bookwormPull (multi-arch manifest)
# Each variant is a multi-arch manifest supporting both amd64 and arm64 docker pull ghcr.io/openhands/agent-server:e6b52e9-pythonRun
All tags pushed for this build
About Multi-Architecture Support
e6b52e9-python) is a multi-arch manifest supporting both amd64 and arm64e6b52e9-python-amd64) are also available if needed