Skip to content

Complete reviewed app integration and native voice source update - #524

Merged
lalalune merged 92 commits into
mainfrom
codex/native-main-pins-current-20261014
Oct 15, 2026
Merged

lalalune merged 92 commits into
mainfrom
codex/native-main-pins-current-20261014

Conversation

@lalalune

@lalalune lalalune commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Complete the reviewed open-PR integration: preserve account/allowance, Maps, emulator instrumentation, separate dictation and interruptible voice, listener invocation, native timing and bounded reset-recovery changes from #521–#523 and their constituent PRs. Also integrate reviewed #528 drawer clearance, #529 local-draft/control cleanup and #530 safe preview entity decoding. Integrate reviewed #527: saved automatic mail-image preference, isolated inline links, and collapsed fallback links not already represented in the message. Preserve latest main #526's Eliza pin fb6232523074e020dacc849cca5ed390d28b8d67 and llama.cpp 56e70ff2a0ca; do not roll back to the earlier #524 pin.

Voice staging now rejects stale source revisions and mismatched artifact hashes. Because the latest main upstream builder does not include elizaOS/eliza#36287's provenance fields, scripts/build-pinned-voice.mjs invokes that unmodified builder from authenticated clean sources, checks identities before and after the build, verifies fresh output hashes, and adds both source revisions to its build record. Unrecorded development staging requires the original matching build record. No vendor checkout is patched. Tests reject stale/missing provenance, wrong artifacts, failed builds and changed source identities.

Keep #526's recorded engine and embedding-JNI candidate hashes. Retain the earlier #524 Paradee-JNI builds as explicitly historical references at 87d9bd96477f. Voice qualification remains false and the old reviewed embedding-host receipt remains refused pending requalification. The native engine gitlink, FFI, JNI and native CMake/build inputs are unchanged between these two Eliza revisions; the shared builder's provenance fields and private agent-event subscriptions differ. Alpha does not yet consume the new event subscription feed. Refresh current inventory/status references and preserve historical evidence boundaries.

Validation for final head 008f1fb:

  • 37 focused reconciliation checks passed: wrapper failure/provenance/artifact tests, documentation consistency, Paradee runtime/staging integrity, and embedding-host refusal.
  • Independent reviewer ran the real resident dependency audit at fb623252: all 98 findings across 3675 versions match latest main; 13 audit tests and the offline audit pass. Existing dispositions are preserved; the removed pacote advisory stays removed.
  • Executed the exact pinned builder and inputs in a temporary layout with Node 24: imports/TypeScript loaded and its invalid-argument path ran as expected, without starting a native build. Four wrapper tests were rerun after input authentication was added.
  • Corrected stale Maps browser navigation expectations; all nine focused cases pass. Replaced the Inbox withdrawal test’s fixed timer turns with awaited readiness; 12 repeated runs and three delayed-WebCrypto runs pass.
  • The Calendar day-column browser test now seeds on a fixed Monday; reproduced the Thursday/weekend failure and passed three fixed-date repeats. Combined email/storage browser suites pass all 13 cases, plus typecheck, HTML isolation and both Inbox fixtures.
  • Prior combined repository verification passes: 1288/1288 tests and production bundle checks.
  • Earlier integration checks covered voice/listener/reset and login-surface/Cloud-sign-in contracts. Prior combined Android workflow passed both standalone and launcher debug/release builds, lint, and flag-off audits. Prior combined resident-source preparation passed. That browser run reported one stale production Settings expectation (268 other cases passed) before concurrent updates cancelled its development shards. The expectation is fixed and the complete nine-case production Settings suite passes on the final candidate, including saved off/on state without Gmail and unreadable preferences failing closed. Broad hosted browser runs remain in progress; no complete final-head browser pass is claimed. Final-head repository verification and Android builds are pending. Release APKs are unsigned and have no packaged resident/voice runtime; they are explicitly non-distributable. No local full Android rebuild or full native build was performed for this reconciliation.

APK builds, emulator HOME-role tests, full AOSP boot, real integrations and device/user acceptance remain separate evidence. This integration does not promote build records into runtime or device acceptance.

Additional review evidence: #528 passes 11 drawer/accessibility cases, three extra short-screen/inset checks and six coverage tests. #529 passes 37 focused cases plus both initial setup timeouts twice on isolated reruns, 14 production voice cases, owning unit checks and typecheck. #530 passes three decoder cases, three rendered cases and both Inbox contracts. Combined tree differs from the reviewed #530 stack only in the deterministic Calendar date and stronger production Settings tests.

Additional local review validation at source tree 008f1fb: all three production Settings tests pass, including image preference persistence and refusal to overwrite unreadable preferences. All seven focused drawer, mail controls and mail preview browser tests pass. All three mail preview unit tests pass. The production bundle audit passes. These browser tests use controlled fixtures and establish no live Gmail or physical-device acceptance. Hosted full checks remain authoritative for the merge head.

lalalune and others added 30 commits October 13, 2026 20:44
…nto claude/cloud-signin-subscription

# Conflicts:
#	test/notice-owner-startup.test.mjs
- Cloud unreachable, 5xx, 429 or no entitlement route is now a closable Offline
  notice (local apps stay, agent waits), never "Choose a plan"; this also covers
  phones updated from the pre-gate build with no admission record.
- Remove the unsigned billing/limits fallback and its allowlist entries; only a
  verified signed entitlement admits the agent.
- Stop the on-device agent at every non-admitting gate even when this renderer
  did not start it.
- Revoke the key a re-approval replaced, once, with the old bearer.
- Reset revokes the Cloud key on the pending-reset launch path too, behind a
  blocking notice.
- Offline grace: only fresh verified entitlements refresh it; ended plan windows
  admit nothing; stale grace stops the agent.
- Redeem: refuse revoked replays, use effectiveUntil, keep the code on failure
  with aria-invalid/describedby, one-time-code autocomplete.
- Throttle return re-checks, recheck on network return and on billing refusals.
- BYO Cerebras key form only in test-mocks builds; denylisted in production.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Adds the native collector for issue #395: AudioRecord timestamps for the
last VAD speech window, the endpoint, local TTS stage marks, first PCM at
MediaPlayer.start and the audible start from the player's output timestamp,
all on CLOCK_MONOTONIC, with source commit and dirty state, engine and model
hashes, device, ABI, build, network and thermal state per trial. Adds an
opt-in emulator harness and campaign driver, v2 aggregation with cold/warm
p50/p95 and acceptance limited to clean release builds on arm64 devices, and
port/repeat support in the emulator audio injector.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ve capture

Transcribe (dictation) records until Stop and fills an editable, unsent draft;
the voice assistant listens continuously, sends each finished spoken turn and
speaks the reply. Both stream the app-process Silero VAD capture to the
resident agent's on-device Phonon-2 live route.

- voice-assistant.ts: VAD endpointing with a short-gap semantic port (no Clef
  in pin c7a9a0d7e7, so the documented long-gap fallback decides), generation
  ids, barge-in that cancels the in-flight request or playback, sentence-level
  heard accounting, mute, Stop speaking, read-reply pauses and binding checks.
- live-segment-stream.ts: one result per VAD segment and stop reconciliation
  (audio after the tap is never forwarded; the open segment ends at the tap).
- native-live-voice.ts: owner-bound live transport, continuous capture, the
  assistant listener (echo-cancelled) and live dictation.
- Android: echo-cancelled communication source for the assistant
  (LiveCaptureSource), and the last in-segment frames are flushed on stop.
- Conversation state keeps only the played part of an interrupted reply.

Refs #423

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Implements the pinned plugin-maps MapsDeviceNavigation contract (select,
start, reroute, stop) over the existing Maps screen state. The agent asks
and the phone decides: start plans from one fresh device fix and stops at
the reviewed route sheet (awaiting-confirmation), so guidance begins only
on the user's Start; reroute uses the screen's own "Reroute from here"
only from a fresh, precise fix. Stale, approximate or missing fixes are
awaiting-location or a typed refusal, never a route origin. A newer
request, Stop, leaving Maps or an account change cancels pending work.
GPS, the active route, guidance and local speech stay on the phone and no
position is returned to the agent. Sharing is unchanged.

The binding is offered only while Eliza Cloud Maps is installed for the
signed-in account, in the launcher renderer, never for the test-mocks
regional gateway; each account gets its own binding and a change
withdraws it first.

The pinned Android resident (c7a9a0d7e7) does not load plugin-maps and
upstream has no relay from MapsService.registerDeviceNavigation to a
phone, so the agent's MAPS_* tools do not reach the phone yet. The port
records the offer (relay: false) and the test fails when the pin starts
loading plugin-maps or registering phone navigation.

Refs #412

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The emulator's virtual microphone delivered only silence on a loaded host
(ranchu HAL pcm_readi failures), so the harness can read the speech WAVs as
a paced feed labelled synthetic-feed while VAD, ASR, TTS and playback stay
the production paths. Fixes from review: harness trials carry their own
pipeline, overlapping replies fail instead of passing, cold flags survive
noise turns, engine loads are counted, builds record a dirty flag, and
acceptance needs a clean release build on an arm64 device.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Speech during a pending or playing reply interrupts only after a 700 ms
  hold or once its transcript passes the disfluency/echo gate; echo, coughs
  and blips leave the reply running. Voice interrupts playback only over a
  capture whose platform echo canceller is attached and enabled (reported by
  startLiveCapture); otherwise Stop speaking remains.
- The echo gate compares against every sentence that has started playing.
- Mute closes a capture that is still opening; unmute waits for the muted
  capture to finish closing.
- A 502 ASR_FAILED segment yields no text and the capture continues; an error
  event and the 502 for one turn count once.
- Frames are stamped on the capture clock and the closing speech-end carries
  the stop time; the renderer waits for the native flush as long as Java.
- A lost Transcribe capture can be started again; Transcribe copy says the
  agent on this phone transcribes and no message is sent.
- Docs state that heard-content marking is this session's view only.

Refs #423

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…edupe

- During playback only an accepted transcript interrupts: residual echo can
  hold the VAD open longer than any fixed hold. The 700 ms hold applies only
  while the request is pending.
- ASR failure dedupe uses per-source counters and only when an event stream
  exists, so consecutive failed segments each yield exactly one result.
- No microphone reopen during an owner-review pause; Stop cancels a muted
  capture that is still finalizing.
- The capture clock never lags wall time by more than the record buffer, and
  the echo effect is released if enabling it throws.

Refs #423

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…its status in Settings

- The assistant surface reads the one-shot invocation when the shell is
  ready; only the listener's authenticated invocation, while the keyguard is
  unlocked (reported by takeInvocation) and the surface is visible and free,
  starts the voice assistant: local Silero VAD, streaming Phonon-2 through the
  resident agent on this phone and Paradee replies (#423). A plain assist
  entry opens the surface without opening the microphone.
- Settings › Character shows the listener's status with Pause listening, Turn
  off listening and Listening settings; nothing in Alpha turns listening on.
  Production speech is described as on this device.
- The ADR and fused-speech doc no longer claim Silero VAD inside the listener
  (its packaged JNI has no VAD entry points) and describe the hand-off.

Refs #424

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Work in progress, interrupted by a rate limit. Not yet compiled or re-run on an emulator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Work in progress, interrupted by a rate limit. Not yet compiled or re-run on an emulator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Work in progress, interrupted by a rate limit. Not yet compiled or re-run on an emulator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Work in progress, interrupted by a rate limit. Not yet compiled or re-run on an emulator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Work in progress, interrupted by a rate limit. Not yet compiled or re-run on an emulator.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tion settles

- AlphaAssistActivity keeps a listener origin for 5 s and remembers whether
  the keyguard was showing when it arrived (AssistInvocation.take), and fires
  alpha:assist-invocation when a new invocation reaches the open surface.
- The renderer reads the origin only with a settled connection, re-checks on
  connection changes, and treats ARIA modal dialogs as blocking.
- Settings orders listener reads after Pause/Turn off and hides Turn off when
  listening is already off.

Refs #424

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Uncommitted edits left in the #467 fix-round worktree when it stopped.
Not verified; see the draft PR for what remains.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@lalalune
lalalune merged commit cca0846 into main Oct 15, 2026
10 of 11 checks passed
@lalalune
lalalune deleted the codex/native-main-pins-current-20261014 branch October 23, 2026 03:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants