diff --git a/CHANGELOG.md b/CHANGELOG.md index 730ac0f9b1..757e1489c3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -21,6 +21,8 @@ Pixel Avatars can be an elf, dwarf, orc or talking flower with beards, medieval ### Added +- Window Companions offer optional, quiet text-paced babble and grab, fast-drag, throw and first-impact landing sounds, off by default for each Profile, and a damped angular spring makes drag reversals and settling continuous without adding model tools or TTS. Idle companions hide the empty status bubble and fade action controls in on hover or keyboard focus; pointer focus after dragging no longer keeps reading open. Long message bubbles follow the latest revealed text without scrollbars; scrolling upward preserves the reading position until returning to the bottom. Dismissing the final card shows waiting replies and lets later arrivals display even while the character retains keyboard focus ([#1173](https://github.com/BotHarness/DeepSeekBot/issues/1173)). + - The CLI can create blank, Zip/directory and GitHub/Git PersonaBots, configure models and inspect Channel, attention and Session summaries through a running Host, retaining known partial results and offering explicit identity cleanup with Memory preserved. See the [CLI guide](docs/bot-cli.md) ([#1363](https://github.com/BotHarness/DeepSeekBot/issues/1363)). - The CLI can authorize Feishu and WeChat IM applications through the existing Provider, return a phone-scannable QR, submit credentials or verification codes through stdin and poll or cancel the same authorization attempt. See the [CLI guide](docs/bot-cli.md) ([#1318](https://github.com/BotHarness/DeepSeekBot/issues/1318)). @@ -79,6 +81,12 @@ Pixel Avatars can be an elf, dwarf, orc or talking flower with beards, medieval ### Fixed +- Window Companion bubbles update their origin together with the character during dragging, keyboard movement and reduced-motion release, without waiting for another animation frame ([#1248](https://github.com/BotHarness/DeepSeekBot/issues/1248)). + +- Reading a Window Companion bubble with the keyboard or resizing the window no longer scrolls the outer stage and leaves the character above the floor; the bubble itself still scrolls ([#1173](https://github.com/BotHarness/DeepSeekBot/issues/1173)). + +- Window Companions keep falling after a fast throw clips the character outside the visible content, and delayed frames no longer slow gravity into an apparent midair pause. Releasing immediately ends the drag, so a late pointer move cannot grab the character again; the release click remains suppressed. Visibility checks follow the fixed presentation area, while hidden pages and offscreen areas still pause animation ([#1173](https://github.com/BotHarness/DeepSeekBot/issues/1173)). + - Opening a Bot DM by Channel ID, including the first Bot-mode entry, now shows that PersonaBot's Channel sidebar and shares its collapse/expansion preferences with Bot navigation; stale conversation metadata no longer selects another Channel's entries ([#1361](https://github.com/BotHarness/DeepSeekBot/issues/1361)). - Clicking the tutorial mask advances to the next step instead of dismissing the tour; the final step uses the Done action. The footer keeps Skip as text and uses labeled icons for Previous, Next and Done so the controls and progress do not wrap ([#1356](https://github.com/BotHarness/DeepSeekBot/issues/1356)). diff --git a/CHANGELOG.zh.md b/CHANGELOG.zh.md index 430a3f87aa..75cafd51d8 100644 --- a/CHANGELOG.zh.md +++ b/CHANGELOG.zh.md @@ -21,6 +21,8 @@ ### Added +- 窗口伙伴新增随文字播放的轻柔叽咕声与抓起、快速拖动、抛出及首次落地音效,按 Profile 保存开关且默认关闭;拖拽倾斜改用阻尼角度弹簧,让反向拖动与回摆连续,不增加模型工具或 TTS。空闲时隐藏占位状态气泡,操作按钮仅在悬浮或键盘聚焦时淡入;拖拽留下的鼠标焦点不再锁住阅读状态。长消息气泡隐藏滚动条并自动跟随最新播放文字;向上翻阅时保持阅读位置,回到底部后恢复跟随。关闭最后一张卡片后立即显示等待中的回复,小人仍保留键盘焦点时,后续新消息也能继续出现([#1173](https://github.com/BotHarness/DeepSeekBot/issues/1173))。 + - CLI 可在运行中的 Host 创建空白、Zip/目录和 GitHub/Git PersonaBot,配置模型并查询 Channel、待办与 Session 摘要;失败保留已知完成结果,显式清理身份时保留 Memory。见 [CLI 指南](docs/bot-cli.md)([#1363](https://github.com/BotHarness/DeepSeekBot/issues/1363))。 - CLI 可经既有 Provider 完成飞书与微信的 IM 应用授权,返回手机扫码二维码,通过 stdin 提交凭据或验证码,并查询或取消同一次授权。参见 [CLI 指南](docs/bot-cli.md)([#1318](https://github.com/BotHarness/DeepSeekBot/issues/1318))。 @@ -79,6 +81,12 @@ ### Fixed +- 窗口伙伴在拖拽、键盘移动与减少动效下松手时,同步更新人物和气泡来源锚点,无需等待下一动画帧([#1248](https://github.com/BotHarness/DeepSeekBot/issues/1248))。 + +- 用键盘阅读窗口伙伴气泡或调整窗口尺寸时,不再滚动外层舞台、让人物悬在底部上方;气泡内部仍可滚动([#1173](https://github.com/BotHarness/DeepSeekBot/issues/1173))。 + +- 窗口伙伴快速甩到可见内容区外后仍会继续下落,绘制帧延迟也不再让重力变慢而看似悬空。松手会立即结束拖拽,迟到的鼠标移动不会把人物重新抓住,同时仍避免松手误开私聊。可见性检测改为跟随固定显示区域,页面隐藏或整个区域离屏时仍暂停动画([#1173](https://github.com/BotHarness/DeepSeekBot/issues/1173))。 + - 通过 Channel ID 打开 Bot 私聊(包括首次进入 Bot 模式)时,现在显示对应 PersonaBot 的侧栏,并与 Bot 导航共享收起与展开偏好;旧对话元数据不会再导致侧栏显示其他 Channel 的条目([#1361](https://github.com/BotHarness/DeepSeekBot/issues/1361))。 - 点击教程遮罩会进入下一步,不再退出引导;最后一步执行完成按钮的动作。底部的跳过保留为文字,上一步、下一步和完成改用带无障碍名称的图标,按钮和进度不再挤压换行([#1356](https://github.com/BotHarness/DeepSeekBot/issues/1356))。 diff --git a/docs/architecture/botharness-architecture.en.md b/docs/architecture/botharness-architecture.en.md index dd31fb167a..71b28d4dff 100644 --- a/docs/architecture/botharness-architecture.en.md +++ b/docs/architecture/botharness-architecture.en.md @@ -642,7 +642,9 @@ The [#1152](https://github.com/BotHarness/BotHarness/issues/1152) QQ group-text If an EventSource enters the terminal CLOSED state after a temporary HTTP failure, the collection owner replaces it with one retry timer backing off from 1 to 8 seconds. The last acknowledged process consumer ID is carried in `?resume=`; native `Last-Event-ID` takes precedence when the replacement subsequently reconnects automatically. Both paths use the same authenticated, bounded Host lease and canonical requalification. Successful baselines reset backoff; removing the last selection or disposing the Plugin cancels the timer and resume identity. No resume identity survives a Client restart. -The Client's independent `CompanionMotion` owns bounded drag poses, velocity sampling, continuous gravity, damped lateral motion, a small floor rebound and settling. View owns the existing frame loop and eases character transforms; bubbles follow the same position. Resizing preserves the current motion and retargets bounds and the drag origin; pointer cancellation matches the active capture. Final landing persists only the normalized horizontal position. Reduced motion returns directly to the floor without pose effects. These are independently implemented presentation dynamics, informed by Coopanion's drag/air/landing interaction rather than imported source or artwork. +The Client's independent `CompanionMotion` owns bounded drag poses, velocity sampling, continuous gravity, damped lateral motion, a small floor rebound and settling. View owns the existing frame loop; a damped second-order angular spring preserves angle and angular velocity through drag reversal, holding and release, while bubbles follow the same position. Bounded integration steps and an angle limit prevent delayed frames from destabilizing the pose. Resizing preserves the current motion and retargets bounds and the drag origin; pointer cancellation matches the active capture. Final landing persists only the normalized horizontal position. Reduced motion clears spring energy and returns directly to the floor without pose effects. These are independently implemented presentation dynamics, informed by Coopanion's drag/air/landing interaction rather than imported source or artwork. + +One Client-owned `CompanionSound` serves the collection, consuming newly revealed committed graphemes from the existing playback loop and grab, fast-drag, fast-release and first-ground-impact feedback from `CompanionMotion`. Drag squeaks have a 500ms cooldown; small rebounds do not repeat the landing sound. Cancellation and reduced-motion resets do not replay feedback. The existing `speechSound` storage field now retains the unified companion-sound preference without resetting saved choices. The global companion setting persists per Client/Profile and defaults off. Web Audio is allocated lazily by a trusted Human gesture; blocked or unavailable audio does not affect message playback. Short synthesized syllables vary with Bot identity and text, skip punctuation/whitespace, and use bounded concurrency and rate limits without a catch-up queue. Muting, hidden/offscreen presentation, stale synchronization, archive, removal and disposal stop the relevant voices. This presentation adds neither model tools nor TTS, and audio resources and playback progress are not persisted ([#1173](https://github.com/BotHarness/DeepSeekBot/issues/1173)). For companion consumers, Avatar exposes a disposable anchor reader. It captures the supported head's local top-center once per Avatar replacement, then reads only that local group's displayed screen matrix in the existing visible-frame loop, including outer rotation, scale and transition origin. Activity pixel covers use the same stable head-region point with the SVG root matrix. Images, unknown rigs and unavailable SVG geometry use the visible Avatar box top-center. There is no whole-SVG bounds scan per frame or Host pose RPC. `CompanionBubbles` keeps the original projected point while placing upright cards within viewport limits, above the toolbar and away from other companions; a pixel-style connector preserves the relationship after displacement. Reading keeps its placement continuity and existing grace period. Hidden/offscreen presentation stops sampling, and Avatar replacement/disposal clears the reader ([#1177](https://github.com/BotHarness/DeepSeekBot/issues/1177)). diff --git a/docs/architecture/botharness-architecture.md b/docs/architecture/botharness-architecture.md index e3c092a756..835e83f609 100644 --- a/docs/architecture/botharness-architecture.md +++ b/docs/architecture/botharness-architecture.md @@ -798,7 +798,9 @@ The [#1152](https://github.com/BotHarness/BotHarness/issues/1152) QQ group-text EventSource 在暂时 HTTP 失败后进入终态 CLOSED 时,集合 owner 用单一重试计时器重新建立连接,间隔从 1 秒退避到最多 8 秒。最后确认的过程消费 ID 通过 `?resume=` 携带;新连接随后原生自动重连时,`Last-Event-ID` 优先于查询参数。两条路径共享同一认证、有界 Host 租约与规范资格重查。成功基线重置退避;移除最后一个选择或卸载 Plugin 清理计时器和恢复标识。Client 重启不保留恢复标识。 -Client 独立的 `CompanionMotion` 拥有有界拖拽姿态、速度采样、连续重力、横向阻尼、轻微地面回弹与落地收敛。View 继续拥有既有帧循环,对人物变换做缓动;气泡跟随同一位置。调整窗口保留当前运动并重新收敛边界、协调拖拽原点,指针取消只匹配活动捕获。最终落地只保存归一化横向位置;减少动效时直接回到底部并关闭姿态效果。这些呈现动力学独立实现,参考 Coopanion 的拖拽/空中/落地交互概念,不引入其源码或美术。 +Client 独立的 `CompanionMotion` 拥有有界拖拽姿态、速度采样、连续重力、横向阻尼、轻微地面回弹与落地收敛。View 继续拥有既有帧循环;带阻尼的二阶角弹簧在拖拽反向、停留与松手时保留角度和角速度,气泡跟随同一位置。有界积分步长和倾角上限避免延迟帧使姿态失稳。调整窗口保留当前运动并重新收敛边界、协调拖拽原点,指针取消只匹配活动捕获。最终落地只保存归一化横向位置;减少动效时清除弹簧能量、直接回到底部并关闭姿态效果。这些呈现动力学独立实现,参考 Coopanion 的拖拽/空中/落地交互概念,不引入其源码或美术。 + +集合共用一个 Client-owned `CompanionSound`,消费既有文字播放循环新显示的已提交字素,以及 `CompanionMotion` 的抓起、快速拖动、快速释放与首次触地反馈。拖动音效冷却 500ms,地面小反弹不重复播放;取消捕获与减少动效不补播。沿用既有 `speechSound` 存储字段保存统一伙伴音效偏好,避免重置已选设置。全局伙伴声音设置按 Client/Profile 持久化,默认关闭;Web Audio 只在可信 Human 操作时按需创建,受限或不可用不影响消息。短合成音节随 Bot 身份和文字变化,标点与空白静音,并发与频率有界,不排队补播。静音、隐藏/离屏、断线、归档、移除和销毁停止相应声音。该呈现不新增模型工具或 TTS,不持久化音频资源或播放进度([#1173](https://github.com/BotHarness/DeepSeekBot/issues/1173))。 Avatar 为伙伴消费者提供可释放的锚点读取器:每次形象替换只采集一次受支持头部的局部顶部中心,既有可见帧循环只读取该局部组当前显示的屏幕矩阵,包含外层旋转、缩放及过渡中的变换原点。像素 Activity 覆盖使用相同头部区域点与 SVG 根矩阵;图片、未知 rig 或不可用的 SVG 几何使用可见 Avatar 盒子顶部中心。不逐帧扫描整个 SVG 边界,也不向 Host 发送姿态 RPC。`CompanionBubbles` 保留原始投射点,让直立卡片受视口、工具栏与多伙伴避让约束,偏移后用像素式连线保持来源关系。阅读延续既有位置与离开宽限;隐藏/离屏停止采样,形象替换/销毁清除读取器([#1177](https://github.com/BotHarness/DeepSeekBot/issues/1177))。 diff --git a/docs/assets/pr/1173-archive-audio/README.md b/docs/assets/pr/1173-archive-audio/README.md new file mode 100644 index 0000000000..3b519a84f6 --- /dev/null +++ b/docs/assets/pr/1173-archive-audio/README.md @@ -0,0 +1,9 @@ +# Actual DSH archive and resume during speech + +Runtime source 4072b9e6b8add5fa7cfab214b8a769c8220f1894, formal pixel-avatar 0.10.0; Chinese, dark theme, 1559 × 865 at scale 1. A new canonical Bot reply drives partial text reveal and native Web Audio. Native pin/sound controls and the owning Host's pause/resume RPC exercise real Bot archive and resume, without fabricating product events or state. + +Before archive, the card is partially revealed and audio starts are observed. Across 120 archived samples and at least five seconds, source starts remain 11 → 11, one immediate stop is observed, cards are absent and the mouth is restored. After resume, starts remain 11 → 11 for at least three seconds with zero historical cards. The same document survives both transitions. Bot and original sound preference are restored; observer and owned browser are closed. No console errors occur. + +A bounded observer delegates actual oscillator/buffer source start and stop calls to the original native methods. Its 23 observed calls do not modify audio output or capture a microphone. This qualifies source admission and cessation, not acoustic playback, every retained resource, genuine background, unsupported AudioContext or autoplay failure. No performance trace or heap measurement ran during this recording. + +The continuous silent VP9 recording contains 466 frames over 31.066 seconds and was container-remuxed only. The PNGs are successive states at one fixed runtime, not a code-version comparison. Chat and roster contain isolated QA fixtures only; no Human-supplied image, login credential, raw profile log or raw browser trace is published. diff --git a/docs/assets/pr/1173-archive-audio/archive-audio-and-resume.webm b/docs/assets/pr/1173-archive-audio/archive-audio-and-resume.webm new file mode 100644 index 0000000000..0042c0a29a Binary files /dev/null and b/docs/assets/pr/1173-archive-audio/archive-audio-and-resume.webm differ diff --git a/docs/assets/pr/1173-archive-audio/archived-no-audio-or-old-cards.png b/docs/assets/pr/1173-archive-audio/archived-no-audio-or-old-cards.png new file mode 100644 index 0000000000..dd24aa7292 Binary files /dev/null and b/docs/assets/pr/1173-archive-audio/archived-no-audio-or-old-cards.png differ diff --git a/docs/assets/pr/1173-archive-audio/qualification.json b/docs/assets/pr/1173-archive-audio/qualification.json new file mode 100644 index 0000000000..489cd669c3 --- /dev/null +++ b/docs/assets/pr/1173-archive-audio/qualification.json @@ -0,0 +1,87 @@ +{ + "source": "4072b9e6b8add5fa7cfab214b8a769c8220f1894", + "formalAvatar": "0.10.0", + "viewport": { + "width": 1559, + "height": 865, + "deviceScaleFactor": 1 + }, + "theme": "dark", + "locale": "zh-CN", + "method": "Actual DSH canonical Bot reply, native pin/sound UI and owning Host pause/resume; bounded observation of actual Web Audio calls during real Bot archive/resume, not audible playback or background qualification", + "canonicalReply": { + "characters": 2921, + "bodySha256": "ddf338096648fda6a71a2a4e9e872164d1f8cd848ea55b60484e67b34d621f7c", + "exactRequestedText": true + }, + "playing": { + "archived": false, + "sync": "live", + "cards": 1, + "mouth": ["closed"], + "sourceStarts": 5, + "immediateStops": 0 + }, + "archived": { + "archived": true, + "sync": "live", + "cards": 0, + "mouth": ["saved"], + "sourceStarts": 11, + "immediateStops": 1 + }, + "archivedHeld": { + "archived": true, + "sync": "live", + "cards": 0, + "mouth": ["saved"], + "sourceStarts": 11, + "immediateStops": 1 + }, + "archivedObservedMilliseconds": 6165.0999999996275, + "archivedSamples": 120, + "resumed": { + "archived": false, + "sync": "live", + "cards": 0, + "mouth": ["saved"], + "sourceStarts": 11, + "immediateStops": 1 + }, + "resumedHeld": { + "archived": false, + "sync": "live", + "cards": 0, + "mouth": ["saved"], + "sourceStarts": 11, + "immediateStops": 1 + }, + "resumedObservedMilliseconds": 4231.300000000745, + "audioObservation": { + "events": 23, + "cap": 2500, + "outputModified": false, + "audiblePlaybackQualified": false + }, + "sameDocument": true, + "cleanup": { + "botResumed": true, + "soundRestored": true, + "observerRemoved": true, + "browserClosed": true + }, + "consoleErrors": 0, + "video": { + "seconds": 31.066, + "frames": 466, + "codec": "VP9", + "silent": true, + "containerRemuxOnly": true + }, + "limits": [ + "Actual foreground Host archive/resume, not genuine background, unsupported AudioContext or autoplay failure.", + "Native audio starts/stops are observed without modifying output; acoustic listening and complete retained-resource disposal are not inferred.", + "Screenshots are successive states of one fixed runtime, not a matched revision before/after pair.", + "No performance trace or heap snapshot was recorded during this flow." + ] +} diff --git a/docs/assets/pr/1173-archive-audio/resumed-no-audio-replay.png b/docs/assets/pr/1173-archive-audio/resumed-no-audio-replay.png new file mode 100644 index 0000000000..e2585e8579 Binary files /dev/null and b/docs/assets/pr/1173-archive-audio/resumed-no-audio-replay.png differ diff --git a/docs/assets/pr/1173-archive-audio/speaking-before-cut.png b/docs/assets/pr/1173-archive-audio/speaking-before-cut.png new file mode 100644 index 0000000000..5e9787b09f Binary files /dev/null and b/docs/assets/pr/1173-archive-audio/speaking-before-cut.png differ diff --git a/docs/assets/pr/1173-companion-audio/README.md b/docs/assets/pr/1173-companion-audio/README.md new file mode 100644 index 0000000000..2bcab0c0f7 --- /dev/null +++ b/docs/assets/pr/1173-companion-audio/README.md @@ -0,0 +1,25 @@ +# Companion interaction audio qualification + +Real isolated DSH 0.2.0-rc.1 at implementation `4776990c2bf82f8dc97578c440afa1fe9b15196f`, Chinese, dark theme, 1559 × 865, existing QA PersonaBot Grace. Screenshots are unchanged browser captures. The continuous silent WebM is an official Chrome DevTools MCP recording, remuxed without re-encoding. + +| Same-build setting: off | Same-build setting: enabled | +| ---------------------------- | ------------------------------- | +| ![](settings-before-off.png) | ![](settings-after-enabled.png) | + +These are actual switch states, not a comparison against the pre-feature build. Earlier before/after feature evidence remains separately identified in PR #1180. + +| Reload before the first gesture | Rest after grabbing and releasing | +| ----------------------------------- | --------------------------------- | +| ![](reloaded-before-first-drag.png) | ![](first-drag-rest.png) | + +The persisted setting survived reload. With no historical companion cards, the first trusted drag admitted Web Audio, generated one grab voice and one landing voice, and ended at rest with character bottom865 matching viewport865. A separate QA-only branch on the actual Web Audio output recorded the sound without a microphone or product-volume change. The audio sample is separate from the silent video; it has not been synchronized or dubbed into that video. The Human listened and explicitly accepted keeping this quiet volume. + +[Continuous silent drag recording](first-drag.webm) · [Actual grab and landing audio sample](grab-and-land.mp3) + +| Muted native drag: rest | Reduced-motion native drag: rest | +| ----------------------- | -------------------------------- | +| ![](muted-rest.png) | ![](reduced-rest.png) | + +The muted and reduced-motion runs generated zero voices and returned to floor865. Reduced motion retained the saved mouth expression. Sound off and system motion were restored after testing. Console inspection found no JavaScript errors; the existing form-field id/name browser issue remains. + +Two real native drags produced grab/landing only. They do not qualify the speed-triggered squeak/whoosh: those paths have automated coverage but still require an actual fast-gesture listening check. This is not a calibrated fast-throw test, mouth acceptance, frame-rate measurement or memory measurement. No full-scenario acceptance is claimed. diff --git a/docs/assets/pr/1173-companion-audio/first-drag-rest.png b/docs/assets/pr/1173-companion-audio/first-drag-rest.png new file mode 100644 index 0000000000..8064f2382a Binary files /dev/null and b/docs/assets/pr/1173-companion-audio/first-drag-rest.png differ diff --git a/docs/assets/pr/1173-companion-audio/first-drag.webm b/docs/assets/pr/1173-companion-audio/first-drag.webm new file mode 100644 index 0000000000..59b595fcc2 Binary files /dev/null and b/docs/assets/pr/1173-companion-audio/first-drag.webm differ diff --git a/docs/assets/pr/1173-companion-audio/grab-and-land.mp3 b/docs/assets/pr/1173-companion-audio/grab-and-land.mp3 new file mode 100644 index 0000000000..f4c6c275be Binary files /dev/null and b/docs/assets/pr/1173-companion-audio/grab-and-land.mp3 differ diff --git a/docs/assets/pr/1173-companion-audio/muted-rest.png b/docs/assets/pr/1173-companion-audio/muted-rest.png new file mode 100644 index 0000000000..a1b35f3f14 Binary files /dev/null and b/docs/assets/pr/1173-companion-audio/muted-rest.png differ diff --git a/docs/assets/pr/1173-companion-audio/reduced-rest.png b/docs/assets/pr/1173-companion-audio/reduced-rest.png new file mode 100644 index 0000000000..cf46beafd7 Binary files /dev/null and b/docs/assets/pr/1173-companion-audio/reduced-rest.png differ diff --git a/docs/assets/pr/1173-companion-audio/reloaded-before-first-drag.png b/docs/assets/pr/1173-companion-audio/reloaded-before-first-drag.png new file mode 100644 index 0000000000..47e63925f1 Binary files /dev/null and b/docs/assets/pr/1173-companion-audio/reloaded-before-first-drag.png differ diff --git a/docs/assets/pr/1173-companion-audio/settings-after-enabled.png b/docs/assets/pr/1173-companion-audio/settings-after-enabled.png new file mode 100644 index 0000000000..0686348ea6 Binary files /dev/null and b/docs/assets/pr/1173-companion-audio/settings-after-enabled.png differ diff --git a/docs/assets/pr/1173-companion-audio/settings-before-off.png b/docs/assets/pr/1173-companion-audio/settings-before-off.png new file mode 100644 index 0000000000..631ce9714a Binary files /dev/null and b/docs/assets/pr/1173-companion-audio/settings-before-off.png differ diff --git a/docs/assets/pr/1173-companion-mouth/README.md b/docs/assets/pr/1173-companion-mouth/README.md new file mode 100644 index 0000000000..93a2cc1c74 --- /dev/null +++ b/docs/assets/pr/1173-companion-mouth/README.md @@ -0,0 +1,17 @@ +# Real mouth reduced-motion qualification + +Captured at4e366904 using formal @botharness/pixel-avatar0.2.0 in real isolated DSH0.2.0-rc.1, Chinese1559×865, dark. A new actual native Bot-authored312-character DM appears through the production companion Feed/reveal. Saved mouth presentation remains unchanged in all411sampled states across idle, actual reduced-motion drag/release,20seconds of message observation and12seconds of settling. The actual drag returns to floor865. Native reduced-motion preference was selected through real settings. No face, text, reply or lifecycle event was injected. + +reduced-idle-dark.png and reduced-message-dark.png show successive idle/message states with the same saved face; they are not before/after implementation comparisons. reduced-mouth.webm is the actual continuous silent official MCP screencast, normal container-remux only. It does not establish audio, calibrated input speed, performance or Human acceptance. Mouth/Avatar/speech driver source is unchanged between4e366904 and4776990c; this capture still retains its original revision attribution. + +Hidden/background, stale/offscreen, unsupported-image/rig and unpin scenes are not qualified by this recording. Raw owner messages and sampling reports stay private. Separate ordinary-message screenshots and videos in PR1180 qualify closed/half-open/open and end restoration. + +## Separate actual bubble-dismissal qualification + +At runtime4776990c, formal pixel-avatar0.2.0, Chinese1559×865/light, one new actual1170-character native Bot reply drives closed/half-open/open/saved presentation. The existing bubble close receives real focus and native Enter during partial reveal. All27subsequent sampled states show one still-mounted companion, zero message cards and saved mouth. speaking-before-dismiss-light.png and saved-after-dismiss-light.png show successive states; character position and Chat history can differ. dismiss-restores-mouth.webm is the complete continuous silent official MCP recording, with no re-encoding/dubbing/interpolation. + +This is local playback dismissal, not cancellation of a native model Turn or Session. An earlier mouse close tool error and a separate combined run losing its page remain excluded. Its failing stop masked the combined run report, so this narrow successful run separately saves each phase and stops cleanly. No cause is assigned to those browser failures, and this keyboard flow does not certify mouse close. See dismiss-qualification.json. Unpin/background/stale/offscreen/fallback and Human visual acceptance remain separate. + +## Native unpin and repin + +At `e4926548` (latest main integration `78aba544`, formal pixel-avatar 0.5.0), a new real canonical 1170-character reply drives partial reveal. Native Channel header unpin removes the Companion; native repin reconnects live with zero cards and the saved mouth, stable for three further seconds. `before-unpin-light.png`, `after-unpin-light.png`, `repinned-saved-light.png` and `unpin-repin.webm` record that path. Continuous silent official MCP recording, remux only. Sanitized assertions are in `unpin-qualification.json`. No synthetic message/event, broad reconnect, new mouth shape, audio or performance acceptance is claimed. diff --git a/docs/assets/pr/1173-companion-mouth/after-unpin-light.png b/docs/assets/pr/1173-companion-mouth/after-unpin-light.png new file mode 100644 index 0000000000..790cf2f8e7 Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/after-unpin-light.png differ diff --git a/docs/assets/pr/1173-companion-mouth/before-unpin-light.png b/docs/assets/pr/1173-companion-mouth/before-unpin-light.png new file mode 100644 index 0000000000..ff6b62137c Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/before-unpin-light.png differ diff --git a/docs/assets/pr/1173-companion-mouth/dismiss-qualification.json b/docs/assets/pr/1173-companion-mouth/dismiss-qualification.json new file mode 100644 index 0000000000..631303b171 --- /dev/null +++ b/docs/assets/pr/1173-companion-mouth/dismiss-qualification.json @@ -0,0 +1,27 @@ +{ + "runtimeCode": "4776990c2bf82f8dc97578c440afa1fe9b15196f", + "package": "@botharness/pixel-avatar@0.2.0", + "kind": "real-isolated-DSH-native-committed-DM", + "viewport": { + "width": 1559, + "height": 865 + }, + "theme": "light", + "canonicalReplies": 1, + "canonicalBodyLength": 1170, + "observedMouthStates": ["saved", "half-open", "closed", "open"], + "actualAction": "Existing bubble close focused, then native Enter; no synthetic click or handler invocation", + "restoration": { + "samples": 27, + "mouth": "saved", + "remainingCards": 0, + "companionStillMounted": true + }, + "recording": "Continuous silent official MCP WebM; normal container remux only", + "limits": [ + "Earlier native mouse close returned an unstructured tool error; no cause or mouse qualification is claimed.", + "An earlier combined dismiss/unpin run lost its page, and stop error masked its in-memory report. It is excluded; this separate successful run saves each phase and stops cleanly.", + "This verifies local bubble playback cancellation, not native model/Session cancellation.", + "Unpin, hidden/background, stale/offscreen, unsupported rigs/images, performance and Human mouth hand-feel acceptance remain separate." + ] +} diff --git a/docs/assets/pr/1173-companion-mouth/dismiss-restores-mouth.webm b/docs/assets/pr/1173-companion-mouth/dismiss-restores-mouth.webm new file mode 100644 index 0000000000..f7a6852327 Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/dismiss-restores-mouth.webm differ diff --git a/docs/assets/pr/1173-companion-mouth/reduced-idle-dark.png b/docs/assets/pr/1173-companion-mouth/reduced-idle-dark.png new file mode 100644 index 0000000000..21023e5174 Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/reduced-idle-dark.png differ diff --git a/docs/assets/pr/1173-companion-mouth/reduced-message-dark.png b/docs/assets/pr/1173-companion-mouth/reduced-message-dark.png new file mode 100644 index 0000000000..07d37583e8 Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/reduced-message-dark.png differ diff --git a/docs/assets/pr/1173-companion-mouth/reduced-mouth.webm b/docs/assets/pr/1173-companion-mouth/reduced-mouth.webm new file mode 100644 index 0000000000..d617b13f51 Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/reduced-mouth.webm differ diff --git a/docs/assets/pr/1173-companion-mouth/reduced-qualification.json b/docs/assets/pr/1173-companion-mouth/reduced-qualification.json new file mode 100644 index 0000000000..976ad0085e --- /dev/null +++ b/docs/assets/pr/1173-companion-mouth/reduced-qualification.json @@ -0,0 +1,36 @@ +{ + "runtimeCode": "4e366904", + "package": "@botharness/pixel-avatar@0.2.0", + "viewport": { + "width": 1559, + "height": 865 + }, + "theme": "dark", + "actualCanonicalBodyLength": 312, + "samples": [ + { + "phase": "reduced-idle", + "count": 1, + "mouthStates": ["saved"] + }, + { + "phase": "reduced-release", + "count": 19, + "mouthStates": ["saved"] + }, + { + "phase": "real-reduced-reveal", + "count": 242, + "mouthStates": ["saved"] + }, + { + "phase": "real-reduced-settled", + "count": 149, + "mouthStates": ["saved"] + } + ], + "limits": [ + "Actual native new DM and reduced-motion drag observed; no hidden/background, unpin or performance claim.", + "Original source revision retained; no Human mouth hand-feel acceptance yet." + ] +} diff --git a/docs/assets/pr/1173-companion-mouth/repinned-saved-light.png b/docs/assets/pr/1173-companion-mouth/repinned-saved-light.png new file mode 100644 index 0000000000..313396e8ff Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/repinned-saved-light.png differ diff --git a/docs/assets/pr/1173-companion-mouth/saved-after-dismiss-light.png b/docs/assets/pr/1173-companion-mouth/saved-after-dismiss-light.png new file mode 100644 index 0000000000..dcf9cfd10a Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/saved-after-dismiss-light.png differ diff --git a/docs/assets/pr/1173-companion-mouth/speaking-before-dismiss-light.png b/docs/assets/pr/1173-companion-mouth/speaking-before-dismiss-light.png new file mode 100644 index 0000000000..c60b983814 Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/speaking-before-dismiss-light.png differ diff --git a/docs/assets/pr/1173-companion-mouth/unpin-qualification.json b/docs/assets/pr/1173-companion-mouth/unpin-qualification.json new file mode 100644 index 0000000000..4e2297d3cf --- /dev/null +++ b/docs/assets/pr/1173-companion-mouth/unpin-qualification.json @@ -0,0 +1,17 @@ +{ + "runtimeCode": "e4926548", + "pixelAvatar": "0.5.0", + "viewport": [1559, 865], + "theme": "light", + "realCanonicalBotReplies": 1, + "bodyLength": 1170, + "checks": [ + "Actual unpin during partial DM reveal removes the companion; native repin reconnects live with saved mouth and zero old cards, no historical replay observed" + ], + "limits": [ + "Continuous silent official Chrome DevTools MCP recording, remux only", + "Native Channel header unpin and repin during partial text reveal", + "No historical replay observed in this capture; not broad reconnect or performance qualification", + "Original formal mouth artwork; does not qualify new opt-in artwork or Human acceptance" + ] +} diff --git a/docs/assets/pr/1173-companion-mouth/unpin-repin.webm b/docs/assets/pr/1173-companion-mouth/unpin-repin.webm new file mode 100644 index 0000000000..78d039ff53 Binary files /dev/null and b/docs/assets/pr/1173-companion-mouth/unpin-repin.webm differ diff --git a/docs/assets/pr/1173-companion-offscreen/README.md b/docs/assets/pr/1173-companion-offscreen/README.md new file mode 100644 index 0000000000..cddb2fdfbf --- /dev/null +++ b/docs/assets/pr/1173-companion-offscreen/README.md @@ -0,0 +1,20 @@ +# Actual offscreen playback and return + +Issue #1173 / PR #1180. Actual integrated runtime: `3dbcce83189294483844e0f396defe6085accac8`, main base `59ab5769`, published `@botharness/pixel-avatar@0.8.0`. The exact built bundle was launched through the isolated DSH helper; authenticated API health passed. Source review against latest-main `652410b5` is separate from runtime integration of that newer Assignment-capacity commit. + +One real Bot `channel_send` produced the exact requested 2,905-character QA message. No fake SSE, product-store mutation, artificial visibility event or direct audio invocation was used. The scene also shows older QA messages in ordinary Chat; they are not the newly revealed companion card. + +The official MCP device-viewport control changes the document from 1559×865 to 1×1, then back. An independent native IntersectionObserver records the character at x=8, y=-95 with intersection ratio zero in that 1×1 viewport. The document stays `visible`. This deliberately extreme viewport establishes the actual nonintersection seam, not ordinary-size layout usability or real background-tab behavior. + +- Before: 204 characters revealed and 95 observed audio-source starts. +- Offscreen interval: 3,989.4ms between observations; 316 characters at both ends, 148 source starts at both ends. +- After return: 408 characters with the old prefix retained; 192 source starts. Same document/time origin, without reload or replay of the committed prefix. +- Console errors: zero. Original sound setting restored, recorder stopped and owned browser closed. + +`visible-before.png` and `visible-after.png` are actual 1559×865 screenshots. Resize clamps the character to the left edge, so this is a behavior pair rather than an identical-position comparison. The upright message bubble and connector are the merged anchor implementation. This does not qualify the separate immediate-release regression in #1248. + +`offscreen-return.webm` is the continuous official MCP silent screencast, losslessly remuxed only: 54.600s, no interpolation, dubbing or speed change. `observed-output.mp3` converts the actual QA-observed Web Audio output branch: 15.576s, without volume adjustment or synthesized replacement audio. The two captures use different clocks/durations; they are separate artifacts and do not establish a synchronized soundtrack or complete wall-clock audio timeline. The source-start observations establish no newly admitted voice while offscreen; this is not a new Human listening approval. + +The first native-window resize attempt never established nonintersection. A second attempt used an incompatible object argument for the official string-based emulation control and stopped before setup. Both failures and cleanup remain preserved privately; neither is passing evidence. The successful run uses the documented viewport string and checks actual geometry. + +The allowlisted `qualification.json` omits private login URLs, cookies, message bodies, Bot/Channel identities and raw logs. Genuine background qualification, calibrated fast-drag/throw feedback and performance acceptance remain separate open gates. diff --git a/docs/assets/pr/1173-companion-offscreen/observed-output.mp3 b/docs/assets/pr/1173-companion-offscreen/observed-output.mp3 new file mode 100644 index 0000000000..c1558f3cfa Binary files /dev/null and b/docs/assets/pr/1173-companion-offscreen/observed-output.mp3 differ diff --git a/docs/assets/pr/1173-companion-offscreen/offscreen-return.webm b/docs/assets/pr/1173-companion-offscreen/offscreen-return.webm new file mode 100644 index 0000000000..84ef625b8f Binary files /dev/null and b/docs/assets/pr/1173-companion-offscreen/offscreen-return.webm differ diff --git a/docs/assets/pr/1173-companion-offscreen/qualification.json b/docs/assets/pr/1173-companion-offscreen/qualification.json new file mode 100644 index 0000000000..bfae8a814d --- /dev/null +++ b/docs/assets/pr/1173-companion-offscreen/qualification.json @@ -0,0 +1,125 @@ +{ + "sourceRevision": "3dbcce83189294483844e0f396defe6085accac8", + "pixelAvatarVersion": "0.8.0", + "method": "Real DSH canonical Bot channel_send; official MCP viewport emulation; independent native IntersectionObserver and QA-only observation of actual Web Audio sources/output.", + "canonicalReply": { + "exactRequestedBody": true, + "characters": 2905, + "verifiedBotAuthor": true + }, + "before": { + "atMs": 82892.69999998808, + "documentVisibility": "visible", + "viewport": { + "width": 1559, + "height": 865 + }, + "characterBox": { + "x": 777.984375, + "y": 769, + "width": 96, + "height": 96, + "top": 769, + "right": 873.984375, + "bottom": 865, + "left": 777.984375 + }, + "revealedCharacters": 204, + "observedSourceStarts": 95, + "audioContextState": "running", + "intersection": { + "visible": true, + "ratio": 1 + } + }, + "offscreenStart": { + "atMs": 91611.5, + "documentVisibility": "visible", + "viewport": { + "width": 1, + "height": 1 + }, + "characterBox": { + "x": 8, + "y": -95, + "width": 96, + "height": 96, + "top": -95, + "right": 104, + "bottom": 1, + "left": 8 + }, + "revealedCharacters": 316, + "observedSourceStarts": 148, + "audioContextState": "running", + "intersection": { + "visible": false, + "ratio": 0 + } + }, + "offscreenEnd": { + "atMs": 95600.90000000596, + "documentVisibility": "visible", + "viewport": { + "width": 1, + "height": 1 + }, + "characterBox": { + "x": 8, + "y": -95, + "width": 96, + "height": 96, + "top": -95, + "right": 104, + "bottom": 1, + "left": 8 + }, + "revealedCharacters": 316, + "observedSourceStarts": 148, + "audioContextState": "running", + "intersection": { + "visible": false, + "ratio": 0 + } + }, + "after": { + "atMs": 101539.40000000596, + "documentVisibility": "visible", + "viewport": { + "width": 1559, + "height": 865 + }, + "characterBox": { + "x": 96.1875, + "y": 769, + "width": 96, + "height": 96, + "top": 769, + "right": 192.1875, + "bottom": 865, + "left": 96.1875 + }, + "revealedCharacters": 408, + "observedSourceStarts": 192, + "audioContextState": "running", + "intersection": { + "visible": true, + "ratio": 1 + } + }, + "offscreenObservedMs": 3989.4000000059605, + "newSourceStartsOffscreen": 0, + "sameDocument": true, + "returnedWithPreservedPrefix": true, + "consoleErrors": [], + "originalSoundSettingRestored": true, + "recordingStopped": true, + "ownedBrowserClosed": true, + "complete": true, + "limits": [ + "The actual offscreen viewport is deliberately 1x1; this qualifies native nonintersection handling, not usability at ordinary window dimensions.", + "Document remains visible throughout; no background visibility, calibrated fast gesture, performance budget or Human listening acceptance is claimed.", + "Continuous silent video and separately captured actual Web Audio output have different capture clocks/durations; no synchronized soundtrack or wall-clock audio completeness is claimed.", + "Earlier failed native-window resize and malformed emulation invocation are preserved privately and excluded from passing evidence." + ] +} diff --git a/docs/assets/pr/1173-companion-offscreen/visible-after.png b/docs/assets/pr/1173-companion-offscreen/visible-after.png new file mode 100644 index 0000000000..380bf276de Binary files /dev/null and b/docs/assets/pr/1173-companion-offscreen/visible-after.png differ diff --git a/docs/assets/pr/1173-companion-offscreen/visible-before.png b/docs/assets/pr/1173-companion-offscreen/visible-before.png new file mode 100644 index 0000000000..94003715bb Binary files /dev/null and b/docs/assets/pr/1173-companion-offscreen/visible-before.png differ diff --git a/docs/assets/pr/1173-companion-reading/README.md b/docs/assets/pr/1173-companion-reading/README.md new file mode 100644 index 0000000000..5f82365cc9 --- /dev/null +++ b/docs/assets/pr/1173-companion-reading/README.md @@ -0,0 +1,15 @@ +# Real long-message reading qualification + +Recorded using official Chrome DevTools MCP in real isolated DSH0.2.0-rc.1, Chinese1559×865. Baseline c396fbab and current4e366904 use identical full message bodies and seeded pixel recipes. Three real native Bot-authored channel_send replies per Profile were independently verified. No Session rewrite, mock SSE or fabricated reply. + +Before-follow-light.png is the valid baseline automatic-follow observation: revealed text stays at scrollTop0 with scrollbar-width:auto. The later baseline manual-focus test failed because the old paragraph was not focusable; that attempted interaction is excluded, and its late final retains only Grace after Ada cards expire. This is not a matched three-card final pair. + +Current after-follow-light.png shows automatic following. In after-history-light.png, native hover expands two Ada cards and the first text paragraph genuinely has keyboard focus. Control+Home keeps that card at0 while reveal grows305→510characters; the other Ada card and Grace card independently advance288→540. Control+End returns the selected completed card to540 in after-tail-light.png. That last action qualifies completed-tail return, not additional output after following resumes. Earlier d6156ff6 evidence in PR1180 separately qualifies resume during output. Cards remain320×104px; text viewport caps at54px, full text remains and scrollbar-width:none. + +independent-reading.webm is the actual continuous silent recording (41.633seconds,1249encoded30fpsframes), container-remuxed without re-encoding, dubbing or interpolation. It is not display-FPS or frame-cost evidence. Screenshots match text/theme/viewport/recipes; character positions, Chat/onboarding backgrounds, model arrivals and exact elapsed reveal differ. Captures are attributed to their original revisions, not current4776990c. New evidence is now repository-hosted after the previous authenticated-upload failure; no GitHub attachment URL is fabricated. + +## Fresh arrivals after final dismissal + +At `4776990c`, a real native reply A appeared; closing its last card with native Enter left the character keyboard-focused and reading active, and real reply B committed but stayed queued with no card. A separate native Tab, without reconnecting, released it. The original red run saved canonical/DOM assertions rather than a matching screenshot or recording. + +`dismiss-fresh-green.webm` is continuous real DSH evidence at `e4926548`, integrated with main `78aba544` and formal pixel-avatar 0.5.0: reply A appears, native Enter closes its last card, and new canonical reply B appears while keyboard focus stays on the character. There is no Tab or reload between dismissal and B. `dismiss-fresh-green.png` shows that fresh card and retained focus. Sanitized red/green verdicts are in `dismiss-fresh-qualification.json`. The fix freezes arrivals only while there are visible cards to preserve; final dismissal also releases already-waiting replies. No new mouth shape, fake SSE, synthetic Bot message, performance or broad reconnect claim is included. Media is silent official MCP output, container remux only. diff --git a/docs/assets/pr/1173-companion-reading/after-follow-light.png b/docs/assets/pr/1173-companion-reading/after-follow-light.png new file mode 100644 index 0000000000..9034b62d8e Binary files /dev/null and b/docs/assets/pr/1173-companion-reading/after-follow-light.png differ diff --git a/docs/assets/pr/1173-companion-reading/after-history-light.png b/docs/assets/pr/1173-companion-reading/after-history-light.png new file mode 100644 index 0000000000..f8e89e5b75 Binary files /dev/null and b/docs/assets/pr/1173-companion-reading/after-history-light.png differ diff --git a/docs/assets/pr/1173-companion-reading/after-tail-light.png b/docs/assets/pr/1173-companion-reading/after-tail-light.png new file mode 100644 index 0000000000..282c2b9f8b Binary files /dev/null and b/docs/assets/pr/1173-companion-reading/after-tail-light.png differ diff --git a/docs/assets/pr/1173-companion-reading/before-follow-light.png b/docs/assets/pr/1173-companion-reading/before-follow-light.png new file mode 100644 index 0000000000..5978e5743e Binary files /dev/null and b/docs/assets/pr/1173-companion-reading/before-follow-light.png differ diff --git a/docs/assets/pr/1173-companion-reading/dismiss-fresh-green.png b/docs/assets/pr/1173-companion-reading/dismiss-fresh-green.png new file mode 100644 index 0000000000..2648f4e1fe Binary files /dev/null and b/docs/assets/pr/1173-companion-reading/dismiss-fresh-green.png differ diff --git a/docs/assets/pr/1173-companion-reading/dismiss-fresh-green.webm b/docs/assets/pr/1173-companion-reading/dismiss-fresh-green.webm new file mode 100644 index 0000000000..9d08858037 Binary files /dev/null and b/docs/assets/pr/1173-companion-reading/dismiss-fresh-green.webm differ diff --git a/docs/assets/pr/1173-companion-reading/dismiss-fresh-qualification.json b/docs/assets/pr/1173-companion-reading/dismiss-fresh-qualification.json new file mode 100644 index 0000000000..bec08559cd --- /dev/null +++ b/docs/assets/pr/1173-companion-reading/dismiss-fresh-qualification.json @@ -0,0 +1,36 @@ +{ + "red": { + "runtimeCode": "4776990c", + "phases": [ + { + "phase": "AFTER_RESTART_A", + "canonical": true, + "visible": true + }, + { + "phase": "AFTER_DISMISS_B", + "canonical": true, + "visible": false + } + ] + }, + "green": { + "runtimeCode": "e4926548", + "formalPixelAvatar": "0.5.0", + "phases": [ + { + "phase": "AFTER_RESTART_A", + "canonical": true, + "visible": true + }, + { + "phase": "AFTER_DISMISS_B", + "canonical": true, + "visible": true + } + ] + }, + "procedure": "Real native reply A → existing close button focused and native Enter → real native reply B with character keyboard focus retained; no Tab, reload or fake event between dismissal and B", + "attribution": "The prior red run saved DOM assertions, not a matching screenshot/video; only the green video is continuous media. A separate native Tab probe released queued B without reconnecting, identifying empty-reader queueing.", + "limits": "Green cold baseline and exact canonical messages were checked; this is not broad reconnect, background, audio or performance acceptance." +} diff --git a/docs/assets/pr/1173-companion-reading/independent-reading.webm b/docs/assets/pr/1173-companion-reading/independent-reading.webm new file mode 100644 index 0000000000..dc6d8b8d81 Binary files /dev/null and b/docs/assets/pr/1173-companion-reading/independent-reading.webm differ diff --git a/docs/assets/pr/1173-position-visual-pair/README.md b/docs/assets/pr/1173-position-visual-pair/README.md new file mode 100644 index 0000000000..48ea8f6cff --- /dev/null +++ b/docs/assets/pr/1173-position-visual-pair/README.md @@ -0,0 +1,15 @@ +# Matched static floor and menu comparison + +The before control and candidate both use checkout `7b1eba8c`, with built production code `774d9f66` and formal Avatar 0.10.0. Only the control's wrapper position is restored to the old left/bottom implementation; the candidate uses translate3d. A test-only geometry adapter also differs in the control and is not bundled. Source and compiled-bundle hashes were checked against the prepared pair. No newer main changes or artwork changes are included. + +Both revisions were launched sequentially through the isolated DSH helper with the same canonical Profile and existing Ada DM. Fresh native Chrome contexts pinned the actual Bot through the Channel header, paused movement and used native ArrowLeft keys to align x=8. Both share the same Avatar recipe hash and the same17message ID/body hash. No new message was injected or replay qualified. Chinese UI,960×640 and actual native light/dark themes match each screenshot pair. + +The static Avatar wrapper measures x8/y544/96×96, ending at the640px viewport floor. Off-character focus lets the toolbar fade to zero opacity. Shift+F10 opens the actual menu in its portal, with matching measured bounds inside the viewport. These checks cover one paused idle Bot and its menu; no general visual-equivalence or other interaction claim. Decoded RGBA hashes in qualification.json describe the captured screenshots only. Natural Avatar blink phase is not artificially controlled. + +The26.233s silent candidate recording continuously shows native menu opening/closing and the theme change. It is container-remuxed only. Raw browser reports and Host authentication remain private. Both owned browsers were closed, the old-layout Host stopped and the candidate Host restored to the same Profile. Console error count is zero in both captures. + +Attempt1 stopped at a conservative process guard: Get-Process reports100ns creation ticks while CIM truncates to microseconds. No Host was stopped and no capture from that attempt is used. The previous failed dark Settings selector remains retained/excluded; the successful capture verifies actual dark rendering directly. No performance profiling was started while the machine was busy. + +To reproduce, start the original-layout control and candidate with the same isolated Profile, using separate fresh browser contexts. At960×640 in Chinese, open Ada's DM, pin through the header, pause walking, focus the character and move left using ArrowLeft until x8. Focus the header Profile chip and check the toolbar fades out, then use Shift+F10 on the character and inspect the menu. Repeat with native dark preference. The source control's exact diff is the wrapper position plus the unbundled test adapter. + +Remaining: five companions, concurrent live messages/reading, dynamic projected anchors, resize, calibrated fast throw comparison, genuine background, unsupported/autoplay and matched performance with the stricter walking budget. Earlier Human motion and sound feedback stays separate. This artifact set does not finish issue1173 or1167 and grants no merge approval. diff --git a/docs/assets/pr/1173-position-visual-pair/candidate-dark-floor.png b/docs/assets/pr/1173-position-visual-pair/candidate-dark-floor.png new file mode 100644 index 0000000000..4333813323 Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/candidate-dark-floor.png differ diff --git a/docs/assets/pr/1173-position-visual-pair/candidate-dark-menu.png b/docs/assets/pr/1173-position-visual-pair/candidate-dark-menu.png new file mode 100644 index 0000000000..ef8c2bf24f Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/candidate-dark-menu.png differ diff --git a/docs/assets/pr/1173-position-visual-pair/candidate-light-floor.png b/docs/assets/pr/1173-position-visual-pair/candidate-light-floor.png new file mode 100644 index 0000000000..61c2d86eb6 Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/candidate-light-floor.png differ diff --git a/docs/assets/pr/1173-position-visual-pair/candidate-light-menu.png b/docs/assets/pr/1173-position-visual-pair/candidate-light-menu.png new file mode 100644 index 0000000000..d2ab34eaa4 Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/candidate-light-menu.png differ diff --git a/docs/assets/pr/1173-position-visual-pair/candidate-menu-and-theme.webm b/docs/assets/pr/1173-position-visual-pair/candidate-menu-and-theme.webm new file mode 100644 index 0000000000..33ab6c7e99 Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/candidate-menu-and-theme.webm differ diff --git a/docs/assets/pr/1173-position-visual-pair/parent-dark-floor.png b/docs/assets/pr/1173-position-visual-pair/parent-dark-floor.png new file mode 100644 index 0000000000..4333813323 Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/parent-dark-floor.png differ diff --git a/docs/assets/pr/1173-position-visual-pair/parent-dark-menu.png b/docs/assets/pr/1173-position-visual-pair/parent-dark-menu.png new file mode 100644 index 0000000000..ef8c2bf24f Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/parent-dark-menu.png differ diff --git a/docs/assets/pr/1173-position-visual-pair/parent-light-floor.png b/docs/assets/pr/1173-position-visual-pair/parent-light-floor.png new file mode 100644 index 0000000000..61c2d86eb6 Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/parent-light-floor.png differ diff --git a/docs/assets/pr/1173-position-visual-pair/parent-light-menu.png b/docs/assets/pr/1173-position-visual-pair/parent-light-menu.png new file mode 100644 index 0000000000..d2ab34eaa4 Binary files /dev/null and b/docs/assets/pr/1173-position-visual-pair/parent-light-menu.png differ diff --git a/docs/assets/pr/1173-position-visual-pair/qualification.json b/docs/assets/pr/1173-position-visual-pair/qualification.json new file mode 100644 index 0000000000..0a7f070296 --- /dev/null +++ b/docs/assets/pr/1173-position-visual-pair/qualification.json @@ -0,0 +1,335 @@ +{ + "scope": "Matched one-Companion static floor and native portalled menu in light/dark themes; no performance or message/audio qualification", + "complete": true, + "checkout": "7b1eba8c76a68d2c30e05f1d0c1c7ab55dc01aa3", + "builtCode": "774d9f66796bdcec0dd7f6361fcb36e10e76fb32", + "runtimeDifference": "Parent restores original left/bottom positioning; candidate uses translate3d. No other production source difference.", + "formalAvatar": "0.10.0", + "viewport": { + "width": 960, + "height": 640 + }, + "locale": "Chinese", + "fixture": { + "recipeHash": "ac7c02b1be96f5e52a3b1952a996f511f6e79126f9a5eb0d4c18300414f93e42", + "messagesHash": "b2f126019a89ef5f02663206aed41b87022161f6aadd1a0d8bdffb547473ee48", + "messageCount": 17 + }, + "method": "Same canonical DSH Profile; fresh Host and isolated native Chrome per revision; native header pin/pause, ArrowLeft alignment, Shift+F10 menu, native dark preference", + "geometryMatched": true, + "pairs": [ + { + "label": "light-floor", + "parent": "parent-light-floor.png", + "candidate": "candidate-light-floor.png", + "rgbaIdentical": true + }, + { + "label": "light-menu", + "parent": "parent-light-menu.png", + "candidate": "candidate-light-menu.png", + "rgbaIdentical": true + }, + { + "label": "dark-floor", + "parent": "parent-dark-floor.png", + "candidate": "candidate-dark-floor.png", + "rgbaIdentical": true + }, + { + "label": "dark-menu", + "parent": "parent-dark-menu.png", + "candidate": "candidate-dark-menu.png", + "rgbaIdentical": true + } + ], + "observations": [ + { + "role": "parent", + "label": "light-floor", + "dark": false, + "sync": "live", + "phase": "rest", + "rect": { + "x": 8, + "y": 544, + "width": 96, + "height": 96, + "top": 544, + "right": 104, + "bottom": 640, + "left": 8 + }, + "walkingPaused": true, + "toolbarOpacity": "0", + "menus": [] + }, + { + "role": "candidate", + "label": "light-floor", + "dark": false, + "sync": "live", + "phase": "rest", + "rect": { + "x": 8, + "y": 544, + "width": 96, + "height": 96, + "top": 544, + "right": 104, + "bottom": 640, + "left": 8 + }, + "walkingPaused": true, + "toolbarOpacity": "0", + "menus": [] + }, + { + "role": "parent", + "label": "light-menu", + "dark": false, + "sync": "live", + "phase": "rest", + "rect": { + "x": 8, + "y": 544, + "width": 96, + "height": 96, + "top": 544, + "right": 104, + "bottom": 640, + "left": 8 + }, + "walkingPaused": true, + "toolbarOpacity": "1", + "menus": [ + { + "rect": { + "x": 72, + "y": 210, + "width": 161.609375, + "height": 297.5, + "top": 210, + "right": 233.609375, + "bottom": 507.5, + "left": 72 + }, + "insideCharacter": false + } + ] + }, + { + "role": "candidate", + "label": "light-menu", + "dark": false, + "sync": "live", + "phase": "rest", + "rect": { + "x": 8, + "y": 544, + "width": 96, + "height": 96, + "top": 544, + "right": 104, + "bottom": 640, + "left": 8 + }, + "walkingPaused": true, + "toolbarOpacity": "1", + "menus": [ + { + "rect": { + "x": 72, + "y": 210, + "width": 161.609375, + "height": 297.5, + "top": 210, + "right": 233.609375, + "bottom": 507.5, + "left": 72 + }, + "insideCharacter": false + } + ] + }, + { + "role": "parent", + "label": "dark-floor", + "dark": true, + "sync": "live", + "phase": "rest", + "rect": { + "x": 8, + "y": 544, + "width": 96, + "height": 96, + "top": 544, + "right": 104, + "bottom": 640, + "left": 8 + }, + "walkingPaused": true, + "toolbarOpacity": "0", + "menus": [] + }, + { + "role": "candidate", + "label": "dark-floor", + "dark": true, + "sync": "live", + "phase": "rest", + "rect": { + "x": 8, + "y": 544, + "width": 96, + "height": 96, + "top": 544, + "right": 104, + "bottom": 640, + "left": 8 + }, + "walkingPaused": true, + "toolbarOpacity": "0", + "menus": [] + }, + { + "role": "parent", + "label": "dark-menu", + "dark": true, + "sync": "live", + "phase": "rest", + "rect": { + "x": 8, + "y": 544, + "width": 96, + "height": 96, + "top": 544, + "right": 104, + "bottom": 640, + "left": 8 + }, + "walkingPaused": true, + "toolbarOpacity": "1", + "menus": [ + { + "rect": { + "x": 72, + "y": 210, + "width": 161.609375, + "height": 297.5, + "top": 210, + "right": 233.609375, + "bottom": 507.5, + "left": 72 + }, + "insideCharacter": false + } + ] + }, + { + "role": "candidate", + "label": "dark-menu", + "dark": true, + "sync": "live", + "phase": "rest", + "rect": { + "x": 8, + "y": 544, + "width": 96, + "height": 96, + "top": 544, + "right": 104, + "bottom": 640, + "left": 8 + }, + "walkingPaused": true, + "toolbarOpacity": "1", + "menus": [ + { + "rect": { + "x": 72, + "y": 210, + "width": 161.609375, + "height": 297.5, + "top": 210, + "right": 233.609375, + "bottom": 507.5, + "left": 72 + }, + "insideCharacter": false + } + ] + } + ], + "consoleErrors": { + "parent": 0, + "candidate": 0 + }, + "closedOwnedBrowsers": { + "parent": true, + "candidate": true + }, + "parentHostStopped": true, + "candidateHostRestored": true, + "video": { + "file": "candidate-menu-and-theme.webm", + "durationSeconds": 26.233, + "silent": true, + "containerRemuxOnly": true + }, + "files": { + "parent-light-floor.png": { + "bytes": 173887, + "sha256": "46732693e711e63c1db9030ba88748cd3878fb620e5c422de347dfd33e4216f7", + "rgbaSha256": "54b1edf2f71d022d6d1af9de804784abf7e4d62f5b0a79889a3ee530524cf953" + }, + "candidate-light-floor.png": { + "bytes": 173887, + "sha256": "46732693e711e63c1db9030ba88748cd3878fb620e5c422de347dfd33e4216f7", + "rgbaSha256": "54b1edf2f71d022d6d1af9de804784abf7e4d62f5b0a79889a3ee530524cf953" + }, + "parent-light-menu.png": { + "bytes": 183682, + "sha256": "3c0b6caf0162fe49ff9c942080a143480bee4166266163227be6aeba02700c25", + "rgbaSha256": "fedd8d290db45416d9a7e18936846eb18d924b1300c54a096bd96ad7cfef198f" + }, + "candidate-light-menu.png": { + "bytes": 183682, + "sha256": "3c0b6caf0162fe49ff9c942080a143480bee4166266163227be6aeba02700c25", + "rgbaSha256": "fedd8d290db45416d9a7e18936846eb18d924b1300c54a096bd96ad7cfef198f" + }, + "parent-dark-floor.png": { + "bytes": 177680, + "sha256": "e2f69e609a9e9dc47a1e0a0e86db57a72c5696d74a9b87dbe3c25fe3be9b9058", + "rgbaSha256": "09deda74a991b2a3a48723ecd110a0be39da6ee3869cae96ddeaeb57c7d99217" + }, + "candidate-dark-floor.png": { + "bytes": 177680, + "sha256": "e2f69e609a9e9dc47a1e0a0e86db57a72c5696d74a9b87dbe3c25fe3be9b9058", + "rgbaSha256": "09deda74a991b2a3a48723ecd110a0be39da6ee3869cae96ddeaeb57c7d99217" + }, + "parent-dark-menu.png": { + "bytes": 186943, + "sha256": "3834699b469f27e7d487027a7769cf47c5e2903ac6a805add8407185ae38375e", + "rgbaSha256": "0b52cfa13749358f3ddf8782ecad07d4059223e2446113949772c2e134dfe619" + }, + "candidate-dark-menu.png": { + "bytes": 186943, + "sha256": "3834699b469f27e7d487027a7769cf47c5e2903ac6a805add8407185ae38375e", + "rgbaSha256": "0b52cfa13749358f3ddf8782ecad07d4059223e2446113949772c2e134dfe619" + }, + "candidate-menu-and-theme.webm": { + "bytes": 2502058, + "sha256": "ea862c7a6ade406496b9a478970e40ef5ce748b85fea1b2b75ce52aedbdfec15" + } + }, + "excluded": [ + "Initial attempt1 failed the guard because Get-Process and CIM creation timestamps differ in sub-microsecond precision; no Host stopped or capture made.", + "Previous missing-dark-option setup is retained and excluded, superseded only for the states captured here." + ], + "remaining": [ + "Five companions, live concurrent messages and reading, dynamic projected anchors, resize and calibrated fast-throw comparison for this candidate.", + "Genuine background and unsupported/autoplay runtime qualification.", + "Matched performance investigation and strict walking p95 budget." + ] +} diff --git a/docs/assets/pr/1173-release-boundary/README.md b/docs/assets/pr/1173-release-boundary/README.md new file mode 100644 index 0000000000..f0573d0cf6 --- /dev/null +++ b/docs/assets/pr/1173-release-boundary/README.md @@ -0,0 +1,11 @@ +# Immediate release boundary evidence + +Both images are generated in the isolated QA instance: before-lift and after-release states of the fixed runtime at the same viewport and theme. They show one interaction sequence, not a matched comparison of code versions. The separately supplied Human failure screenshot remains private and is not included. + +The **after** screenshot and silent video run actual DSH on source `4072b9e6b8add5fa7cfab214b8a769c8220f1894`, integrated main `bcae2205d7562f3eec1d9b2f5e20c1b4fbb5a9da`, formal Avatar0.10.0, Chinese/dark/1559×865. Official Chrome DevTools MCP native drag lifts Ada toward the fixed Bot-mode entry and releases it. This is an ordinary tool-driven lift/drop, **not** a calibrated fast Human fling. A passive bounded observer records trusted pointer-up, fall, floor contact after 1313ms and rest after 1538ms, with no post-release drag, zero console errors and final bottom0px. The observer and owned browser are closed. The video is container-remuxed only; no frames are redrawn or re-timed. + +The Human separately accepted the actual loaded `0eb841dd` upward throw/fall after its release handler was verified. That physical session has no saved video: post-test retrieval failed after page closure/reconnection. The integrated release code is unchanged. Failed recorder setup attempts are excluded. + +Automated regression at the actual component seam replays release → lost capture → late movement → click before the old click timer. It goes red on the original implementation and green with immediate active-pointer clearing; the click remains suppressed and the real motion controller reaches the floor. A deterministic browser-before image would require the same late native pointer timing, which the official drag tool cannot calibrate. The private Human failure image is omitted; these fixed-runtime states and recording must not be described as a matched experiment between revisions. + +For manual comparison: run the preceding revision `98edf282` and this revision through the isolated DSH launcher using the same QA Profile; pin Ada, quickly fling upward and release, then move the unpressed mouse without grabbing again. Check that release never returns to drag, inertia continues and the character rests at the floor. No private login links or conversation payloads are included in qualification.json. Remaining whole-issue background/stale/fallback qualification is independent. diff --git a/docs/assets/pr/1173-release-boundary/after-release-floor.png b/docs/assets/pr/1173-release-boundary/after-release-floor.png new file mode 100644 index 0000000000..ce0ae0e1c4 Binary files /dev/null and b/docs/assets/pr/1173-release-boundary/after-release-floor.png differ diff --git a/docs/assets/pr/1173-release-boundary/before-lift.png b/docs/assets/pr/1173-release-boundary/before-lift.png new file mode 100644 index 0000000000..ffd5286cb1 Binary files /dev/null and b/docs/assets/pr/1173-release-boundary/before-lift.png differ diff --git a/docs/assets/pr/1173-release-boundary/native-lift-and-floor.webm b/docs/assets/pr/1173-release-boundary/native-lift-and-floor.webm new file mode 100644 index 0000000000..12737325c9 Binary files /dev/null and b/docs/assets/pr/1173-release-boundary/native-lift-and-floor.webm differ diff --git a/docs/assets/pr/1173-release-boundary/qualification.json b/docs/assets/pr/1173-release-boundary/qualification.json new file mode 100644 index 0000000000..096dec072a --- /dev/null +++ b/docs/assets/pr/1173-release-boundary/qualification.json @@ -0,0 +1,35 @@ +{ + "source": "4072b9e6b8add5fa7cfab214b8a769c8220f1894", + "main": "bcae2205d7562f3eec1d9b2f5e20c1b4fbb5a9da", + "formalAvatar": "0.10.0", + "viewport": { + "width": 1559, + "height": 865 + }, + "locale": "zh", + "theme": "dark", + "method": "Official Chrome DevTools MCP native tool drag; ordinary lift/drop, not calibrated fast Human throw; silent continuous screencast", + "trustedRelease": true, + "postReleaseDrag": false, + "floorContactMs": 1313, + "restMs": 1538, + "final": { + "phase": "rest", + "bottom": "0px" + }, + "consoleErrors": 0, + "observerRemoved": true, + "ownedBrowserClosed": true, + "humanAcceptance": { + "source": "0eb841dd1265d36a5f9c8399497347c2bf90ce67", + "loadedHandlerVerified": true, + "upwardThrowAccepted": true, + "videoRecorded": false + }, + "before": { + "kind": "Before-lift state of the fixed runtime; not a pre-fix failure image", + "sourceIndependentlyVerified": true, + "matchedViewportAndTheme": true, + "comparison": "Within-version interaction states" + } +} diff --git a/docs/assets/pr/1173-sound-requests/A-settled-B-draft-preserved-dark.png b/docs/assets/pr/1173-sound-requests/A-settled-B-draft-preserved-dark.png new file mode 100644 index 0000000000..92e2f7ce2e Binary files /dev/null and b/docs/assets/pr/1173-sound-requests/A-settled-B-draft-preserved-dark.png differ diff --git a/docs/assets/pr/1173-sound-requests/B-chat-and-companion-light.png b/docs/assets/pr/1173-sound-requests/B-chat-and-companion-light.png new file mode 100644 index 0000000000..a186949353 Binary files /dev/null and b/docs/assets/pr/1173-sound-requests/B-chat-and-companion-light.png differ diff --git a/docs/assets/pr/1173-sound-requests/B-chat-answered-both-settled-light.png b/docs/assets/pr/1173-sound-requests/B-chat-answered-both-settled-light.png new file mode 100644 index 0000000000..f5a9324915 Binary files /dev/null and b/docs/assets/pr/1173-sound-requests/B-chat-answered-both-settled-light.png differ diff --git a/docs/assets/pr/1173-sound-requests/README.md b/docs/assets/pr/1173-sound-requests/README.md new file mode 100644 index 0000000000..299593d135 --- /dev/null +++ b/docs/assets/pr/1173-sound-requests/README.md @@ -0,0 +1,34 @@ +# Actual sound and live requests integration + +Source `50b44535a8180a138268c4f52a7fb5581f61de13`, incorporating main +`d55d82fe`; formal `@botharness/pixel-avatar@0.8.0`. Chinese, 1559 × 865, +one isolated DSH Profile. These media qualify the sound branch with the +already-merged request cards, rather than the new mouth artwork. + +Two real native `ask_user_question` requests contain three questions for A +and one for B. Native inputs select one choice, two checkboxes and a custom +answer. Switching light/dark preserves both drafts. A answers through its +Companion while an unrelated Channel remains selected; B's draft survives. +B then answers through the canonical Chat card, with unique input IDs +across the simultaneously rendered surfaces. Both answers commit exactly +once, continue their original Bot, and reconcile the Companion in the same +document. A late duplicate is refused. See `qualification.json`. + +The screenshots show consecutive actual states, not a before/after code +comparison. The three-question card scrolls within its bounded region; +the dark draft image is scrolled to the custom answer. Question-mark avatars +are actual pending Bot appearances, not placeholders inserted for the capture. + +`two-bot-and-chat.webm` is a continuous 56.833-second silent official Chrome +DevTools MCP screencast, container-remuxed only. `actual-output.mp3` is the +real Web Audio output, observed through a QA-only recording destination +without microphone access or product gain changes. Its separate clock +starts before the screencast; the two files are not synchronized. + +Observed source starts increase from 4 after native grab/landing to 77 after +the original Bots reply. This establishes renewed text-paced sound after +request resolution, not a calibrated fast-gesture or GPU measurement. +There are no browser console errors. The original sound preference is +restored, all request decisions are canonical, and the owned browser is +closed. The failed initial hidden-toolbar attempt is retained privately and +excluded from these results. diff --git a/docs/assets/pr/1173-sound-requests/actual-output.mp3 b/docs/assets/pr/1173-sound-requests/actual-output.mp3 new file mode 100644 index 0000000000..67c4ffd9fa Binary files /dev/null and b/docs/assets/pr/1173-sound-requests/actual-output.mp3 differ diff --git a/docs/assets/pr/1173-sound-requests/drafts-dark.png b/docs/assets/pr/1173-sound-requests/drafts-dark.png new file mode 100644 index 0000000000..ee292d3cec Binary files /dev/null and b/docs/assets/pr/1173-sound-requests/drafts-dark.png differ diff --git a/docs/assets/pr/1173-sound-requests/drafts-light.png b/docs/assets/pr/1173-sound-requests/drafts-light.png new file mode 100644 index 0000000000..9efc74a95d Binary files /dev/null and b/docs/assets/pr/1173-sound-requests/drafts-light.png differ diff --git a/docs/assets/pr/1173-sound-requests/qualification.json b/docs/assets/pr/1173-sound-requests/qualification.json new file mode 100644 index 0000000000..3a50ec70f5 --- /dev/null +++ b/docs/assets/pr/1173-sound-requests/qualification.json @@ -0,0 +1,54 @@ +{ + "source": "50b44535a8180a138268c4f52a7fb5581f61de13", + "formalPixelAvatar": "0.8.0", + "viewport": { + "width": 1559, + "height": 865 + }, + "locale": "zh-CN", + "questionCounts": [3, 1], + "canonicalResolutions": { + "A": 1, + "B": 1 + }, + "AContinuesOriginalBot": true, + "BContinuesOriginalBot": true, + "AAnswers": [ + { + "id": "route", + "selected": ["体验版"] + }, + { + "id": "checks", + "selected": ["拖拽", "气泡"] + }, + { + "id": "note", + "selected": [], + "custom": "保持轻微音量,优先保证阅读稳定。" + } + ], + "BAnswers": [ + { + "id": "scope", + "selected": ["只检查自己"] + } + ], + "unrelatedChannelPreserved": true, + "BCompanionDraftPreserved": true, + "sameDocument": true, + "lateDuplicateRejected": true, + "audioBeforeReplies": { + "events": 4, + "contextState": "running" + }, + "audioAfterReplies": { + "events": 77, + "contextState": "running" + }, + "observedSources": 77, + "outputSampleRate": 48000, + "consoleErrors": 0, + "originalSoundRestored": true, + "ownedBrowserClosed": true +} diff --git a/docs/assets/pr/1173-sound-requests/two-bot-and-chat.webm b/docs/assets/pr/1173-sound-requests/two-bot-and-chat.webm new file mode 100644 index 0000000000..eac0a115c8 Binary files /dev/null and b/docs/assets/pr/1173-sound-requests/two-bot-and-chat.webm differ diff --git a/docs/assets/pr/1173-sound-requests/two-pending-light.png b/docs/assets/pr/1173-sound-requests/two-pending-light.png new file mode 100644 index 0000000000..2faca48642 Binary files /dev/null and b/docs/assets/pr/1173-sound-requests/two-pending-light.png differ diff --git a/docs/assets/pr/1173-stage-focus/README.md b/docs/assets/pr/1173-stage-focus/README.md new file mode 100644 index 0000000000..9641cab2c8 --- /dev/null +++ b/docs/assets/pr/1173-stage-focus/README.md @@ -0,0 +1,9 @@ +# Native keyboard reading and fixed stage + +Before source5cddec35 versus source1c0651ef; DSH0.2.0-rc.1, formal Avatar0.10.0, native dark/English,960×640 CSS viewport, DPR1.425. Both idle screenshots use five real pinned Bots, paused at the same saved horizontal positions. + +The before stage uses hidden overflow. After keyboard reading and resize, Ada's stage retains scrollTop4.912px although physical y and CSS bottom are0; character bottom is635.088px. Clip overflow removes the scroll container. All five after stages remain atscrollTop0, with character bottoms640px; at760×540 the bottoms are540.351px within1px device rounding. + +Real UI composer requests reach the native model and channel_send; no injected Activity or message state. Text grows in the54px bubble, follows the bottom without a scrollbar, and Control+Home/End reads earlier/latest text. The after recording uses a fixed viewport and is copied byte-for-byte from T3; it successfully decodes without malformed frames. Two recordings that resized mid-capture contained malformed H.264 frames and remain private/excluded. Resize is qualified by actual screenshots and geometry instead. See qualification.json for scopes and exclusions. + +Restart deliberately interrupts the owned Host. Its earlier document's connection-lost/shell-timeout is retained; fresh navigation works. No performance, cold-browser, genuine-background or all-motion acceptance is claimed. diff --git a/docs/assets/pr/1173-stage-focus/after-five.png b/docs/assets/pr/1173-stage-focus/after-five.png new file mode 100644 index 0000000000..d1ba5b41e0 Binary files /dev/null and b/docs/assets/pr/1173-stage-focus/after-five.png differ diff --git a/docs/assets/pr/1173-stage-focus/after-reading-resize.png b/docs/assets/pr/1173-stage-focus/after-reading-resize.png new file mode 100644 index 0000000000..0eb56aba84 Binary files /dev/null and b/docs/assets/pr/1173-stage-focus/after-reading-resize.png differ diff --git a/docs/assets/pr/1173-stage-focus/after-reading.mp4 b/docs/assets/pr/1173-stage-focus/after-reading.mp4 new file mode 100644 index 0000000000..dcf8139ad3 Binary files /dev/null and b/docs/assets/pr/1173-stage-focus/after-reading.mp4 differ diff --git a/docs/assets/pr/1173-stage-focus/before-five.png b/docs/assets/pr/1173-stage-focus/before-five.png new file mode 100644 index 0000000000..fc38cdcfa5 Binary files /dev/null and b/docs/assets/pr/1173-stage-focus/before-five.png differ diff --git a/docs/assets/pr/1173-stage-focus/before-reading-top.png b/docs/assets/pr/1173-stage-focus/before-reading-top.png new file mode 100644 index 0000000000..001a75bac2 Binary files /dev/null and b/docs/assets/pr/1173-stage-focus/before-reading-top.png differ diff --git a/docs/assets/pr/1173-stage-focus/before-stream-tail.png b/docs/assets/pr/1173-stage-focus/before-stream-tail.png new file mode 100644 index 0000000000..a3965e7ebb Binary files /dev/null and b/docs/assets/pr/1173-stage-focus/before-stream-tail.png differ diff --git a/docs/assets/pr/1173-stage-focus/qualification.json b/docs/assets/pr/1173-stage-focus/qualification.json new file mode 100644 index 0000000000..da8d3a2ac4 --- /dev/null +++ b/docs/assets/pr/1173-stage-focus/qualification.json @@ -0,0 +1,171 @@ +{ + "issue": 1173, + "beforeSource": "5cddec3541a3c2869412520d42d4ea6dee75caaa", + "afterSource": "1c0651ef8485b4e8a359959bbed14e30b0acf41c", + "afterClientSha256": "57a6b695e5f22e3d7fd93efbf7b320c7ceb4c340ff627d84b9b906c17a3828b6", + "formalPixelAvatar": "0.10.0", + "dsh": "0.2.0-rc.1", + "locale": "English native UI, Chinese canonical test messages", + "theme": "native dark", + "cssViewport": [960, 640], + "dpr": 1.4249999523162842, + "geometryBefore": { + "observations": [ + { + "bot": "阅读验收 Ada · Window Companion", + "bottom": 635.0877151489258, + "overflow": "hidden", + "scrollTop": 4.912281036376953, + "x": 330.4364013671875 + }, + { + "bot": "阅读验收 Grace · Window Companion", + "bottom": 640.0000076293945, + "overflow": "hidden", + "scrollTop": 0, + "x": 404.4211730957031 + }, + { + "bot": "声音与问答验收 A · Window Companion", + "bottom": 640.0000076293945, + "overflow": "hidden", + "scrollTop": 0, + "x": 275.51617431640625 + }, + { + "bot": "声音与问答验收 B · Window Companion", + "bottom": 640.0000076293945, + "overflow": "hidden", + "scrollTop": 0, + "x": 146.61300659179688 + }, + { + "bot": "抛掷回归 Bot · Window Companion", + "bottom": 640.0000076293945, + "overflow": "hidden", + "scrollTop": 0, + "x": 543.6257934570312 + } + ], + "pass": false, + "viewport": [960, 640] + }, + "geometryAfter": { + "card": [ + { + "height": 54, + "length": 431, + "scroll": 396, + "scrollbar": "none", + "top": 341.75439453125 + } + ], + "observations": [ + { + "bot": "阅读验收 Ada · Window Companion", + "bottom": 640.0000076293945, + "overflow": "clip", + "scrollTop": 0, + "x": 330.4364013671875 + }, + { + "bot": "阅读验收 Grace · Window Companion", + "bottom": 640.0000076293945, + "overflow": "clip", + "scrollTop": 0, + "x": 404.4211730957031 + }, + { + "bot": "声音与问答验收 A · Window Companion", + "bottom": 640.0000076293945, + "overflow": "clip", + "scrollTop": 0, + "x": 275.51617431640625 + }, + { + "bot": "声音与问答验收 B · Window Companion", + "bottom": 640.0000076293945, + "overflow": "clip", + "scrollTop": 0, + "x": 146.61300659179688 + }, + { + "bot": "抛掷回归 Bot · Window Companion", + "bottom": 640.0000076293945, + "overflow": "clip", + "scrollTop": 0, + "x": 543.6257934570312 + } + ], + "pass": true, + "viewport": [960, 640] + }, + "readingResizeAfter": { + "card": [ + { + "height": 54, + "length": 402, + "scroll": 378, + "scrollbar": "none", + "top": 0 + } + ], + "observations": [ + { + "bot": "阅读验收 Ada · Window Companion", + "bottom": 540.3508987426758, + "overflow": "clip", + "scrollTop": 0 + }, + { + "bot": "阅读验收 Grace · Window Companion", + "bottom": 540.3508987426758, + "overflow": "clip", + "scrollTop": 0 + }, + { + "bot": "声音与问答验收 A · Window Companion", + "bottom": 540.3508987426758, + "overflow": "clip", + "scrollTop": 0 + }, + { + "bot": "声音与问答验收 B · Window Companion", + "bottom": 540.3508987426758, + "overflow": "clip", + "scrollTop": 0 + }, + { + "bot": "抛掷回归 Bot · Window Companion", + "bottom": 540.3508987426758, + "overflow": "clip", + "scrollTop": 0 + } + ], + "pass": true, + "viewport": [760, 540] + }, + "recordings": [ + { + "file": "after-reading.mp4", + "original": [ + { + "codec_type": "video", + "width": 1368, + "height": 912 + } + ], + "durationSeconds": 38.3555, + "publishedBytes": 7299793, + "encoding": "Original T3 recording, byte-for-byte copy; fixed960×640 CSS viewport; no audio track; decoder validated without malformed frames" + } + ], + "limits": [ + "Two original T3 recordings changed dimensions during capture and contained malformed H.264 frames; excluded and preserved privately. The published fixed-viewport recording was decoded successfully and copied without edits. Resize qualification uses PNG/DOM evidence.", + "Five native paused companions, keyboard navigation, canonical model/channel_send message playback and real resize only. Not calibrated fast throw, moving hover, genuine background, audio or formal 0.10.1 qualification.", + "No profiler or performance acceptance under high machine CPU load.", + "A restart-era document recorded connection-lost and shell-timeout before mounting later; it remains a failed attempt. A separate fresh document mounted the shell in686ms, then native pins/composer/messages were checked independently. This is not a recovery-budget measurement.", + "Actual complete console read contains older GitHub404s, two unsuccessful asset-path probes and intentional owned Host restart connection errors; no new post-navigation exception observed.", + "The before/after idle pair uses the same five saved positions and profile. A new real message is used after restart; historical messages are not replayed." + ] +} diff --git a/docs/assets/pr/1173-stale-audio/README.md b/docs/assets/pr/1173-stale-audio/README.md new file mode 100644 index 0000000000..be3ca9437c --- /dev/null +++ b/docs/assets/pr/1173-stale-audio/README.md @@ -0,0 +1,9 @@ +# Actual DSH stale audio and native removal/repin + +Runtime source4072b9e6b8add5fa7cfab214b8a769c8220f1894, formal pixel-avatar0.10.0; Chinese/dark/1559×865×1. A newly committed native Bot reply drives partial text reveal and actual Web Audio. Native controls enable sound, remove and repin the illustrated Bot. A task-owned loopback forwarding hop destroys the real Companion feed and refuses reconnects for the stale observation, then reconnects; product event/state is never fabricated. + +A bounded QA observer delegates actual oscillator/buffer start and stop calls to their unmodified native methods and records counts; it does not change audio output, force context state or record the microphone. The exact149 observed calls are summarized in qualification.json; no timing, acoustic-output, simultaneous-resource or background acceptance is inferred from screenshots. No trace/heap measurement was running during recording. + +Source starts remain26→26 throughout at least5seconds stale while the same readable card grows; return resumes the same prefix and new syllables. Native removal and repin keep starts74→74 with zero old cards. One immediate source stop was observed; this bounded observation does not independently count every retained owner resource. The same document survives reconnect. The two console errors are expected INCOMPLETE_CHUNKED_ENCODING/EMPTY_RESPONSE from the intentional transport cut; no other errors occur. + +The continuous49.866s/748frame VP9 recording is silent and container-remuxed only. The four PNGs are fixed-runtime sequence states, not a matched code-version pair. Historical Chat/roster contains only the isolated QA fixtures; no Human-supplied image, login token, cookie, raw profile log or raw browser trace is included. Sound preference is restored and observer, owned browser and forwarding hop are closed. A separate --disable-webaudio native-flag probe did not actually disable AudioContext and is excluded from fallback qualification. Genuine background and unsupported/autoplay failure remain separate gates. diff --git a/docs/assets/pr/1173-stale-audio/qualification.json b/docs/assets/pr/1173-stale-audio/qualification.json new file mode 100644 index 0000000000..89a48c32dc --- /dev/null +++ b/docs/assets/pr/1173-stale-audio/qualification.json @@ -0,0 +1,92 @@ +{ + "source": "4072b9e6b8add5fa7cfab214b8a769c8220f1894", + "formalAvatar": "0.10.0", + "viewport": { + "width": 1559, + "height": 865, + "deviceScaleFactor": 1 + }, + "theme": "dark", + "locale": "zh-CN", + "method": "Actual DSH canonical Bot reply, native UI actions, real companion transport interruption; bounded observation of actual Web Audio calls, not audible playback or background qualification", + "canonicalReply": { + "characters": 3099, + "bodySha256": "26d7d04417508f8a5f95b957dd964f9c9ae3ec9c81c4bb12a61ab7d65f42cb8d", + "exactRequestedText": true + }, + "playing": { + "revealedCharacters": 26, + "cards": 1, + "sync": "live", + "sourceStarts": 10, + "immediateStops": 0, + "contextState": "running" + }, + "stale": { + "revealedCharacters": 83, + "cards": 1, + "sync": "stale", + "sourceStarts": 26, + "immediateStops": 1, + "contextState": "running" + }, + "staleHeld": { + "revealedCharacters": 198, + "cards": 1, + "sync": "stale", + "sourceStarts": 26, + "immediateStops": 1, + "contextState": "running" + }, + "staleObservedMilliseconds": 6150.0999999996275, + "recovered": { + "revealedCharacters": 255, + "cards": 1, + "sync": "live", + "sourceStarts": 36, + "immediateStops": 1, + "contextState": "running" + }, + "removed": { + "exists": false, + "sourceStarts": 74, + "heldSourceStarts": 74 + }, + "repinned": { + "sync": "live", + "cards": 0, + "sourceStarts": 74, + "heldSourceStarts": 74 + }, + "audioObservation": { + "events": 149, + "cap": 2500, + "outputModified": false, + "audiblePlaybackQualified": false + }, + "transport": { + "streamsDropped": 1, + "intentionalConsoleErrors": 2, + "unexpectedConsoleErrors": 0 + }, + "sameDocument": true, + "cleanup": { + "soundRestored": true, + "observerRemoved": true, + "browserClosed": true, + "proxyClosed": true + }, + "video": { + "seconds": 49.866, + "frames": 748, + "codec": "VP9", + "silent": true, + "containerRemuxOnly": true + }, + "limits": [ + "Foreground actual Companion feed interruption, not genuine background or whole-network outage.", + "Audio source starts/stops are observed actual native Web Audio calls, not acoustic output or listening acceptance.", + "Screenshots are consecutive states at one fixed runtime, not matched revision before/after.", + "Unsupported AudioContext and autoplay failure are covered by separate regressions, not qualified as real hardware failures here." + ] +} diff --git a/docs/assets/pr/1173-stale-audio/repinned-no-audio-replay.png b/docs/assets/pr/1173-stale-audio/repinned-no-audio-replay.png new file mode 100644 index 0000000000..e247d70411 Binary files /dev/null and b/docs/assets/pr/1173-stale-audio/repinned-no-audio-replay.png differ diff --git a/docs/assets/pr/1173-stale-audio/resumed-same-card.png b/docs/assets/pr/1173-stale-audio/resumed-same-card.png new file mode 100644 index 0000000000..35c2ddf47e Binary files /dev/null and b/docs/assets/pr/1173-stale-audio/resumed-same-card.png differ diff --git a/docs/assets/pr/1173-stale-audio/speaking-before-cut.png b/docs/assets/pr/1173-stale-audio/speaking-before-cut.png new file mode 100644 index 0000000000..9350916494 Binary files /dev/null and b/docs/assets/pr/1173-stale-audio/speaking-before-cut.png differ diff --git a/docs/assets/pr/1173-stale-audio/stale-audio-and-repin.webm b/docs/assets/pr/1173-stale-audio/stale-audio-and-repin.webm new file mode 100644 index 0000000000..aa5ff3c0ba Binary files /dev/null and b/docs/assets/pr/1173-stale-audio/stale-audio-and-repin.webm differ diff --git a/docs/assets/pr/1173-stale-audio/stale-silent-readable.png b/docs/assets/pr/1173-stale-audio/stale-silent-readable.png new file mode 100644 index 0000000000..d9c7b17591 Binary files /dev/null and b/docs/assets/pr/1173-stale-audio/stale-silent-readable.png differ diff --git a/docs/assets/pr/1173-translated-position/README.md b/docs/assets/pr/1173-translated-position/README.md new file mode 100644 index 0000000000..61795305f9 --- /dev/null +++ b/docs/assets/pr/1173-translated-position/README.md @@ -0,0 +1,11 @@ +# Translated-position candidate: native lift and floor return + +Actual source `774d9f66796bdcec0dd7f6361fcb36e10e76fb32`, formal Avatar 0.10.0, Chinese/light/960 × 640. The isolated DSH Host supplied real Bot and Activity data; native Chrome showed live synchronization. The two screenshots are consecutive states of the **same fixed candidate runtime**, before an ordinary native-tool lift and after release, not a before/after revision comparison. Existing long QA messages remain in the Chat; no new message or audio is qualified here. + +The continuous 6.567-second silent WebM is container-remuxed only. A bounded DOM observer recorded trusted pointer-up, subsequent fall, floor contact and rest without returning to drag. Final physical height is zero and the wrapper bottom equals the 640px viewport floor; no native console errors. The observer was removed and the owned native browser was closed. Hashes and allowlisted observations are in qualification.json; raw traces, browser reports, authentication and Human screenshots remain private. + +A later dark-theme setup failed because the expected Settings option was absent. It is excluded, with no dark screenshot or qualification. T3 preview requests timed out although direct authenticated Host reads and native Chrome reads succeeded. This records an environment difference, not a diagnosed or repaired T3 defect. + +No new Human fast-throw approval, calibrated gesture, matched baseline capture, GPU saving, frame-time budget or overall visual-equivalence claim. The Human's earlier motion acceptance persists independently. The CPU remains busy after container cleanup; matched performance collection waits for idle. + +To review, launch the candidate with the isolated DSH helper; use Chinese/light/960 × 640, enter Bot mode, select a pixel Bot and pin it from the Channel header. Drag the character toward the sidebar and release. Check that gravity reaches the viewport floor, then repeat native dark theme, reversal/fast throw, five companions, message reading and projected anchors. Those broader checks remain open for this candidate. diff --git a/docs/assets/pr/1173-translated-position/after-release-floor.png b/docs/assets/pr/1173-translated-position/after-release-floor.png new file mode 100644 index 0000000000..b4008f0bad Binary files /dev/null and b/docs/assets/pr/1173-translated-position/after-release-floor.png differ diff --git a/docs/assets/pr/1173-translated-position/before-lift.png b/docs/assets/pr/1173-translated-position/before-lift.png new file mode 100644 index 0000000000..57b97d7eb4 Binary files /dev/null and b/docs/assets/pr/1173-translated-position/before-lift.png differ diff --git a/docs/assets/pr/1173-translated-position/native-lift-and-floor.webm b/docs/assets/pr/1173-translated-position/native-lift-and-floor.webm new file mode 100644 index 0000000000..40beae69e0 Binary files /dev/null and b/docs/assets/pr/1173-translated-position/native-lift-and-floor.webm differ diff --git a/docs/assets/pr/1173-translated-position/qualification.json b/docs/assets/pr/1173-translated-position/qualification.json new file mode 100644 index 0000000000..7a51f23d3e --- /dev/null +++ b/docs/assets/pr/1173-translated-position/qualification.json @@ -0,0 +1,50 @@ +{ + "source": "774d9f66796bdcec0dd7f6361fcb36e10e76fb32", + "formalAvatar": "0.10.0", + "viewport": { + "width": 960, + "height": 640 + }, + "locale": "Chinese", + "theme": "light", + "method": "Actual DSH; official native Chrome MCP ordinary drag; DOM phase observer, not a performance measurement or calibrated fast gesture", + "complete": true, + "activitySync": "live", + "trustedPointerUp": true, + "firstFallAfterReleaseMs": 5, + "firstFloorAfterReleaseMs": 664, + "restAfterReleaseMs": 1264, + "returnedToDragAfterRelease": false, + "finalFloor": { + "physicalY": 0, + "wrapperBottom": 640, + "viewportHeight": 640 + }, + "consoleErrors": 0, + "observerRemoved": true, + "closedOwnedBrowser": true, + "video": { + "durationSeconds": 6.567, + "silent": true, + "containerRemuxOnly": true + }, + "excluded": [ + "Dark capture failed because the expected Settings option was not present; no dark qualification.", + "T3 preview requests timed out while native Chrome and direct Host calls worked.", + "No calibrated fast gesture, matched before/after revision capture or performance measurement." + ], + "files": { + "before-lift.png": { + "bytes": 173976, + "sha256": "f99899bf44fbca1e92b8ba873615c520ba0856d69ec4657953f27b419f16b948" + }, + "after-release-floor.png": { + "bytes": 171859, + "sha256": "2bdaaebec46fff395756d185dd52adfb37db8261017c9c414011f49db2640715" + }, + "native-lift-and-floor.webm": { + "bytes": 796292, + "sha256": "fb07a6c0465befc415a8ac8f3293bf9b79445905635020d570d03bc8111fdd24" + } + } +} diff --git a/docs/assets/pr/1248-companion-release-anchor/README.md b/docs/assets/pr/1248-companion-release-anchor/README.md new file mode 100644 index 0000000000..f680d67d60 --- /dev/null +++ b/docs/assets/pr/1248-companion-release-anchor/README.md @@ -0,0 +1,27 @@ +# Direct-motion anchor synchronization + +Follow-up #1248 to merged #1183 and parent #1177. Real DSH 0.2.0-rc.1, formal pixel-avatar0.8.0, Chinese,1559×865. Baseline runtime4dbaa; corrected runtimefbb29752. The latter has an identical complete tracked tree to follow-up426167c9 from merged main85e9dfbe. New evidence commits do not change runtime code. + +## Actual reduced-motion release + +| Before, settled | After, settled | +| ------------------------------------ | -------------------------------------- | +| ![](release-before-settled-dark.png) | ![](reduced-released-reading-dark.png) | + +These PNGs do not show the transient defect. A read-only QA observer compares the actual displayed head top-center and SVG connector origin in the first rAF callback after a trusted native pointerup. Before: character bottom865 but connector still points to screen y32 instead of769 (737px error). After: both origins match at y769 (0px). Both later120-sample reading/keyboard/theme/release workloads pass, illustrating why settled checks alone missed the defect. A mounted public View regression independently fails before the fix with300px error and passes after, without executing a subsequent animation frame. + +[Continuous baseline recording](release-before.webm) · [Continuous corrected recording](release-after.webm) · [Sanitized qualification](qualification.json) + +## Corrected reading and keyboard movement + +| Light reading, genuine tool cover during initial reply | Light after native arrow keys | Dark reading | +| ------------------------------------------------------ | ------------------------------- | ----------------------------- | +| ![](reduced-reading-light.png) | ![](reduced-keyboard-light.png) | ![](reduced-reading-dark.png) | + +Two exact new Bot-authored replies are independently checked against canonical Channel messages. Normal public character focus retains the reading list; native pause controls stop autonomous walking. Both original absent Avatar Appearance/image fields remain unchanged. All four phases stay in one document; cards remain upright and within the viewport. Console inspection finds0JavaScript errors and all owned test pages/browser are closed. + +## Boundaries and preserved failures + +Same Profile, identities, history, viewport, theme and scenario; reply marker and sidebar unread count differ. The first baseline setup timed out waiting for visible cards despite real replies. That attempt is retained privately. The next baseline adds live-stream and outside-focus preconditions, so recovery does not prove its cause. It exposes the737px first-release failure; the corrected run then passes the same scenario. + +Recordings are continuous silent official MCP output, container-remuxed only. No synthetic reply/SSE/card injection, interpolation, dubbed audio or invented first-frame image. Sampling does not prove all compositor frames, performance, held/reversing drag, moving-hover/grace or genuine background behavior. Those broader #1177/#1167 gates remain separate. Quiet-idle presentation belongs to independent #1173. diff --git a/docs/assets/pr/1248-companion-release-anchor/qualification.json b/docs/assets/pr/1248-companion-release-anchor/qualification.json new file mode 100644 index 0000000000..675f27922a --- /dev/null +++ b/docs/assets/pr/1248-companion-release-anchor/qualification.json @@ -0,0 +1,120 @@ +{ + "baselineRuntime": "4dbaa0ad07e1d00b9e3d51ad4e2427552bfa509c", + "correctedRuntime": "fbb29752abb56c02ece77c9e99c4c68285ec6fb7", + "equivalentFollowupCommit": "426167c98efe09c2f9c1bbe85f0d8ea0aba02979", + "followupBase": "85e9dfbee4e966049eea4a5e847b114764008e80", + "originalIntegratedMain": "66d2767a065af49afb999f3792d8754a50fc66ff", + "formalPixelAvatar": "0.8.0", + "viewport": [1559, 865], + "locale": "zh-CN", + "themes": ["light", "dark"], + "before": { + "phases": [ + { + "label": "light-reduced-reading", + "samples": 30, + "maxOriginErrorPx": 0, + "allCardsUpright": true + }, + { + "label": "keyboard-moved-reduced", + "samples": 30, + "maxOriginErrorPx": 0, + "allCardsUpright": true + }, + { + "label": "dark-reduced-reading", + "samples": 30, + "maxOriginErrorPx": 0, + "allCardsUpright": true + }, + { + "label": "native-reduced-release-reading", + "samples": 30, + "maxOriginErrorPx": 0, + "allCardsUpright": true + } + ], + "firstObservedFrameAfterRelease": { + "policy": "reduce", + "phase": "rest", + "reading": "true", + "characterBottom": 865, + "viewportHeight": 865, + "expectedScreenPoint": { + "x": 745.484375, + "y": 769 + }, + "connectorScreenPoint": { + "x": 745.484375, + "y": 32 + }, + "originErrorPx": 737, + "visibleCards": 2, + "actualTrustedPointerup": true + }, + "closedOwnedBrowser": true + }, + "after": { + "phases": [ + { + "label": "light-reduced-reading", + "samples": 30, + "maxOriginErrorPx": 0, + "allCardsUpright": true + }, + { + "label": "keyboard-moved-reduced", + "samples": 30, + "maxOriginErrorPx": 0, + "allCardsUpright": true + }, + { + "label": "dark-reduced-reading", + "samples": 30, + "maxOriginErrorPx": 0, + "allCardsUpright": true + }, + { + "label": "native-reduced-release-reading", + "samples": 30, + "maxOriginErrorPx": 0, + "allCardsUpright": true + } + ], + "firstObservedFrameAfterRelease": { + "policy": "reduce", + "phase": "rest", + "reading": "true", + "characterBottom": 865, + "viewportHeight": 865, + "expectedScreenPoint": { + "x": 745.484375, + "y": 769 + }, + "connectorScreenPoint": { + "x": 745.484375, + "y": 769 + }, + "originErrorPx": 0, + "visibleCards": 2, + "actualTrustedPointerup": true + }, + "exactCanonicalBotReplies": 2, + "originalAppearanceUnchanged": true, + "sameDocument": true, + "consoleErrors": 0, + "closedOwnedBrowser": true + }, + "limits": [ + "Native official MCP drag/release and keyboard operations; two newly generated canonical Bot replies, no synthetic presentation, replies or SSE.", + "Normal public character focus retains reading cards; walking paused through native controls; OS reduced-motion preference selected at browser startup.", + "Dark before/after PNGs are settled states and do not show the transient failure. The rAF observer catches the first observed callback after trusted pointerup, not every compositor frame.", + "Same Profile, identities, history, viewport, theme and scenario; reply marker and sidebar unread count differ. Initial light capture includes a genuine transient tool cover.", + "Before: 120 later samples passed despite a 737px first-release mismatch. After: 120 samples plus the first release observation pass. This is bounded geometry qualification, not performance or proof about unsampled frames.", + "An earlier initial-setup attempt timed out waiting for two visible cards. Its native replies and failed capture are retained privately; no root cause is claimed. The next baseline attempt added both live-stream preconditions and outside focus before arrival, so recovery does not establish which change mattered.", + "Before runtime 4dbaa was merged by PR1183 while the correction was being prepared. The correction was moved to a dedicated follow-up from that squash merge; complete tracked trees fbb29752 and 426167c9 are identical.", + "Continuous silent official MCP recordings, remux only; no audio, frame interpolation or invented first-frame screenshot.", + "Held/reversing drag, moving hover/grace and genuine background qualification remain under1177; performance remains under1167. This fix does not qualify those scenarios." + ] +} diff --git a/docs/assets/pr/1248-companion-release-anchor/reduced-keyboard-light.png b/docs/assets/pr/1248-companion-release-anchor/reduced-keyboard-light.png new file mode 100644 index 0000000000..a4968a5f00 Binary files /dev/null and b/docs/assets/pr/1248-companion-release-anchor/reduced-keyboard-light.png differ diff --git a/docs/assets/pr/1248-companion-release-anchor/reduced-reading-dark.png b/docs/assets/pr/1248-companion-release-anchor/reduced-reading-dark.png new file mode 100644 index 0000000000..8ec42cc37f Binary files /dev/null and b/docs/assets/pr/1248-companion-release-anchor/reduced-reading-dark.png differ diff --git a/docs/assets/pr/1248-companion-release-anchor/reduced-reading-light.png b/docs/assets/pr/1248-companion-release-anchor/reduced-reading-light.png new file mode 100644 index 0000000000..638da021a6 Binary files /dev/null and b/docs/assets/pr/1248-companion-release-anchor/reduced-reading-light.png differ diff --git a/docs/assets/pr/1248-companion-release-anchor/reduced-released-reading-dark.png b/docs/assets/pr/1248-companion-release-anchor/reduced-released-reading-dark.png new file mode 100644 index 0000000000..d7d58bb47e Binary files /dev/null and b/docs/assets/pr/1248-companion-release-anchor/reduced-released-reading-dark.png differ diff --git a/docs/assets/pr/1248-companion-release-anchor/release-after.webm b/docs/assets/pr/1248-companion-release-anchor/release-after.webm new file mode 100644 index 0000000000..63d27363e3 Binary files /dev/null and b/docs/assets/pr/1248-companion-release-anchor/release-after.webm differ diff --git a/docs/assets/pr/1248-companion-release-anchor/release-before-settled-dark.png b/docs/assets/pr/1248-companion-release-anchor/release-before-settled-dark.png new file mode 100644 index 0000000000..427fd7deac Binary files /dev/null and b/docs/assets/pr/1248-companion-release-anchor/release-before-settled-dark.png differ diff --git a/docs/assets/pr/1248-companion-release-anchor/release-before.webm b/docs/assets/pr/1248-companion-release-anchor/release-before.webm new file mode 100644 index 0000000000..e976980c0c Binary files /dev/null and b/docs/assets/pr/1248-companion-release-anchor/release-before.webm differ diff --git a/packages/client/src/client/companion-motion.ts b/packages/client/src/client/companion-motion.ts index 8e035cb09a..577a0968d2 100644 --- a/packages/client/src/client/companion-motion.ts +++ b/packages/client/src/client/companion-motion.ts @@ -7,6 +7,11 @@ export interface CompanionPoint { phase: 'rest' | 'drag' | 'fall' | 'land'; } +export interface CompanionInteraction { + kind: 'grab' | 'drag' | 'throw' | 'land'; + strength: number; +} + export class CompanionMotion { point: CompanionPoint = { x: 0, y: 0, width: 0, tilt: 0, squash: 0, phase: 'rest' }; private height = 0; @@ -14,12 +19,23 @@ export class CompanionMotion { private vy = 0; private sampledAt = 0; private landing = 0; + private tiltVelocity = 0; + private tiltTarget = 0; + private dragIdle = 0; + private dragging = false; + private landingFeedback = false; + private dragFeedbackAt = -Infinity; + + constructor(private readonly interaction?: (event: CompanionInteraction) => void) {} measure(width: number, height: number, position: number): CompanionPoint { this.height = Math.max(0, height - 120); this.vx = 0; this.vy = 0; this.landing = 0; + this.dragging = false; + this.landingFeedback = false; + this.resetTilt(); return (this.point = { x: this.clampX(position * Math.max(0, width - 104), width), y: 0, @@ -52,11 +68,22 @@ export class CompanionMotion { } this.sampledAt = now; this.landing = 0; + this.dragIdle = 0; + this.tiltTarget = reduced ? 0 : Math.max(-18, Math.min(18, this.vx / 35)); + if (!reduced) { + if (!this.dragging) this.interaction?.({ kind: 'grab', strength: 1 }); + else if (Math.abs(this.vx) > 500 && now - this.dragFeedbackAt >= 500) { + this.interaction?.({ kind: 'drag', strength: Math.min(1, Math.abs(this.vx) / 700) }); + this.dragFeedbackAt = now; + } + } + this.dragging = true; + if (reduced) this.resetTilt(); return (this.point = { ...this.point, x: nextX, y: nextY, - tilt: reduced ? 0 : Math.max(-18, Math.min(18, this.vx / 35)), + tilt: reduced ? 0 : this.point.tilt, squash: reduced ? 0 : -0.055, phase: 'drag', }); @@ -66,7 +93,18 @@ export class CompanionMotion { this.vx = 0; this.vy = 0; this.sampledAt = now; - return (this.point = { ...this.point, tilt: 0, squash: reduced ? 0 : -0.055, phase: 'drag' }); + this.dragIdle = 0; + this.dragging = false; + this.landingFeedback = false; + this.dragFeedbackAt = -Infinity; + this.tiltTarget = 0; + if (reduced) this.resetTilt(); + return (this.point = { + ...this.point, + tilt: reduced ? 0 : this.point.tilt, + squash: reduced ? 0 : -0.055, + phase: 'drag', + }); } release(now: number, reduced: boolean, cancelled = false): CompanionPoint { @@ -75,7 +113,9 @@ export class CompanionMotion { this.vy = 0; } if (reduced) return this.measure(this.point.width, this.height + 120, this.position()); - if (this.point.y === 0 && this.vx === 0 && this.vy === 0) return this.move(this.point.x); + this.landingFeedback = this.dragging && !cancelled; + if (this.landingFeedback && Math.hypot(this.vx, this.vy) >= 700) + this.interaction?.({ kind: 'throw', strength: 1 }); return (this.point = { ...this.point, phase: 'fall' }); } @@ -85,22 +125,26 @@ export class CompanionMotion { walking: boolean, direction: number, ): CompanionPoint { - if (this.point.phase === 'drag') { - if (reduced && (this.point.tilt || this.point.squash)) - return (this.point = { ...this.point, tilt: 0, squash: 0 }); - return this.point; - } if (reduced) { + this.resetTilt(); + if (this.point.phase === 'drag') return (this.point = { ...this.point, tilt: 0, squash: 0 }); if (this.point.phase !== 'rest') return this.measure(this.point.width, this.height + 120, this.position()); + if (this.point.tilt || this.point.squash) + return (this.point = { ...this.point, tilt: 0, squash: 0 }); return this.point; } - let remaining = Math.min(64, Math.max(0, milliseconds)); + const limit = this.point.phase === 'fall' || this.point.phase === 'land' ? 2000 : 64; + let remaining = Number.isFinite(milliseconds) ? Math.min(limit, Math.max(0, milliseconds)) : 0; let { x, y, tilt, squash, phase } = this.point; while (remaining > 0) { - const dt = Math.min(16, remaining) / 1000; + const dt = Math.min(8, remaining) / 1000; remaining -= dt * 1000; - if (phase === 'fall') { + let target = 0; + if (phase === 'drag') { + this.dragIdle += dt * 1000; + target = this.dragIdle > 80 ? 0 : this.tiltTarget; + } else if (phase === 'fall') { this.vy -= 1900 * dt; this.vx *= Math.exp(-2.5 * dt); x += this.vx * dt; @@ -116,6 +160,10 @@ export class CompanionMotion { if (y <= 0) { y = 0; const impact = Math.abs(this.vy); + if (this.landingFeedback) { + this.landingFeedback = false; + this.interaction?.({ kind: 'land', strength: Math.min(1, impact / 1200) }); + } this.vy = impact > 100 ? impact * 0.24 : 0; squash = Math.min(0.12, impact / 9000); if (!this.vy) { @@ -123,23 +171,31 @@ export class CompanionMotion { this.landing = 220; } } - tilt += (this.vx / 45 - tilt) * (1 - Math.exp(-9 * dt)); + target = Math.max(-18, Math.min(18, this.vx / 45)); squash *= Math.exp(-12 * dt); } else if (phase === 'land') { this.vx *= Math.exp(-16 * dt); x = this.clampX(x + this.vx * dt); - tilt *= Math.exp(-18 * dt); squash *= Math.exp(-18 * dt); this.landing -= dt * 1000; if (this.landing <= 0) { phase = 'rest'; - tilt = 0; squash = 0; this.vx = 0; } } else if (walking) { x = this.clampX(x + direction * 18 * dt); } + this.tiltVelocity += (90 * (target - tilt) - 9 * this.tiltVelocity) * dt; + tilt += this.tiltVelocity * dt; + if (Math.abs(tilt) > 24) { + tilt = Math.max(-24, Math.min(24, tilt)); + if (tilt * this.tiltVelocity > 0) this.tiltVelocity = 0; + } + if (target === 0 && Math.abs(tilt) < 0.015 && Math.abs(this.tiltVelocity) < 0.03) { + tilt = 0; + this.tiltVelocity = 0; + } } if ( x === this.point.x && @@ -155,6 +211,9 @@ export class CompanionMotion { move(x: number): CompanionPoint { this.vx = 0; this.vy = 0; + this.dragging = false; + this.landingFeedback = false; + this.resetTilt(); return (this.point = { ...this.point, x: this.clampX(x), @@ -172,4 +231,9 @@ export class CompanionMotion { private clampX(x: number, width = this.point.width): number { return Math.max(8, Math.min(Math.max(8, width - 104), x)); } + private resetTilt(): void { + this.tiltVelocity = 0; + this.tiltTarget = 0; + this.dragIdle = 0; + } } diff --git a/packages/client/src/client/companion-sound.ts b/packages/client/src/client/companion-sound.ts new file mode 100644 index 0000000000..b24ab39361 --- /dev/null +++ b/packages/client/src/client/companion-sound.ts @@ -0,0 +1,206 @@ +import type { CompanionCard } from './window-companion.js'; +import { companionMessageIdentity } from '../../../core/src/companions/sources.js'; +import type { CompanionInteraction } from './companion-motion.js'; + +export function companionBabble( + before: readonly CompanionCard[], + after: readonly CompanionCard[], +): string | undefined { + for (const card of [...after].reverse()) { + const identity = companionMessageIdentity(card.channelId, card.messageId); + const prior = before.find( + (item) => companionMessageIdentity(item.channelId, item.messageId) === identity, + ); + if (!prior || card.shown <= prior.shown) continue; + const index = card.boundaries.indexOf(card.shown); + const text = card.body.slice(card.boundaries[index - 1], card.shown); + if (/[\p{L}\p{N}\p{Extended_Pictographic}]/u.test(text)) return text; + } + return; +} + +interface Voice { + botId: string; + kind: 'speech' | 'interaction'; + source: AudioScheduledSourceNode; + gain: GainNode; + filter?: BiquadFilterNode; +} + +interface Tone { + type: OscillatorType | 'noise'; + from: number; + to: number; + duration: number; + volume: number; +} + +export class CompanionSound { + private enabled = false; + private context: AudioContext | undefined; + private voices = new Set(); + private lastAt = -Infinity; + private lastByBot = new Map(); + private noise: AudioBuffer | undefined; + + setEnabled(enabled: boolean): void { + this.enabled = enabled; + if (!enabled) this.stop(); + } + + unlock(): void { + if (!this.enabled) return; + try { + this.context ??= typeof AudioContext === 'undefined' ? undefined : new AudioContext(); + if (this.context?.state === 'suspended') void this.context.resume().catch(() => undefined); + } catch {} + } + + play(botId: string, text: string): void { + const context = this.context; + if (!this.enabled || !context || context.state !== 'running') return; + const now = context.currentTime; + if ( + this.voices.size >= 2 || + now - this.lastAt < 0.03 || + now - (this.lastByBot.get(botId) ?? -Infinity) < 0.075 + ) + return; + let identity = 0; + for (const character of botId) identity = (identity * 31 + character.codePointAt(0)!) >>> 0; + const pitch = 170 + (identity % 7) * 16 + ((text.codePointAt(0) ?? 0) % 11) * 9; + if ( + this.synthesize(botId, 'speech', { + type: 'square', + from: pitch, + to: pitch * 0.88, + duration: 0.05, + volume: 0.018, + }) + ) { + this.lastAt = now; + this.lastByBot.set(botId, now); + } + } + + interact(botId: string, event: CompanionInteraction): void { + if (!this.enabled || this.context?.state !== 'running') return; + this.stop(botId); + const tones: Record = { + grab: { type: 'triangle', from: 680, to: 1200, duration: 0.11, volume: 0.018 }, + drag: { type: 'triangle', from: 900, to: 1300, duration: 0.07, volume: 0.01 }, + throw: { type: 'noise', from: 350, to: 2000, duration: 0.16, volume: 0.018 }, + land: { + type: 'triangle', + from: 110, + to: 45, + duration: 0.09, + volume: 0.008 + Math.max(0, Math.min(1, event.strength)) * 0.014, + }, + }; + this.synthesize(botId, 'interaction', tones[event.kind]); + } + + private synthesize(botId: string, kind: Voice['kind'], tone: Tone): boolean { + const context = this.context; + if (!this.enabled || context?.state !== 'running' || this.voices.size >= 2) return false; + const now = context.currentTime; + let source: AudioScheduledSourceNode | undefined; + let gain: GainNode | undefined; + let filter: BiquadFilterNode | undefined; + try { + if (tone.type === 'noise') { + const buffer = + this.noise ?? + context.createBuffer(1, Math.ceil(context.sampleRate * 0.18), context.sampleRate); + if (!this.noise) { + const samples = buffer.getChannelData(0); + for (let index = 0; index < samples.length; index++) + samples[index] = Math.random() * 2 - 1; + this.noise = buffer; + } + const noise = context.createBufferSource(); + source = noise; + noise.buffer = buffer; + filter = context.createBiquadFilter(); + filter.type = 'bandpass'; + filter.Q.setValueAtTime(1.4, now); + filter.frequency.setValueAtTime(tone.from, now); + filter.frequency.exponentialRampToValueAtTime(tone.to, now + tone.duration); + source.connect(filter); + } else { + const oscillator = context.createOscillator(); + source = oscillator; + oscillator.type = tone.type; + oscillator.frequency.setValueAtTime(tone.from, now); + oscillator.frequency.exponentialRampToValueAtTime(tone.to, now + tone.duration); + } + gain = context.createGain(); + const voice: Voice = { botId, kind, source, gain, ...(filter ? { filter } : {}) }; + gain.gain.setValueAtTime(0.0001, now); + gain.gain.exponentialRampToValueAtTime(tone.volume, now + 0.006); + gain.gain.exponentialRampToValueAtTime(0.0001, now + tone.duration); + (filter ?? source).connect(gain); + gain.connect(context.destination); + source.onended = () => this.finish(voice); + this.voices.add(voice); + source.start(now); + source.stop(now + tone.duration + 0.01); + return true; + } catch { + try { + source?.stop(); + } catch {} + for (const voice of this.voices) { + if (voice.source === source) { + this.finish(voice); + return false; + } + } + source?.disconnect(); + filter?.disconnect(); + gain?.disconnect(); + return false; + } + } + + stopSpeech(botId: string): void { + for (const voice of [...this.voices]) { + if (voice.botId === botId && voice.kind === 'speech') this.stopVoice(voice); + } + } + + stop(botId?: string): void { + for (const voice of [...this.voices]) { + if (botId !== undefined && voice.botId !== botId) continue; + this.stopVoice(voice); + } + if (botId === undefined) { + this.lastAt = -Infinity; + this.lastByBot.clear(); + } else this.lastByBot.delete(botId); + } + + private finish(voice: Voice): void { + voice.source.onended = null; + voice.source.disconnect(); + voice.filter?.disconnect(); + voice.gain.disconnect(); + this.voices.delete(voice); + } + + private stopVoice(voice: Voice): void { + try { + voice.source.stop(); + } catch {} + this.finish(voice); + } + + dispose(): void { + this.setEnabled(false); + const context = this.context; + this.context = undefined; + this.noise = undefined; + if (context && context.state !== 'closed') void context.close().catch(() => undefined); + } +} diff --git a/packages/client/src/client/locale.ts b/packages/client/src/client/locale.ts index cdbc16b3f5..9c67a9eecf 100644 --- a/packages/client/src/client/locale.ts +++ b/packages/client/src/client/locale.ts @@ -210,6 +210,9 @@ export const zh = { 'companion.scope.shared': '我与 Bot 都在的会话', 'companion.scope.all-bot': 'Bot 加入的所有会话', 'companion.settings': '窗口伙伴', + 'companion.speechSound': '伙伴音效', + 'companion.speechSoundHint': + '随新消息文字播放轻柔的叽咕声,并为抓起、快速拖动、抛出和落地添加音效。不朗读正文;默认关闭,适用于本 Profile 的所有窗口伙伴。', 'companion.capacityHint': '本 Profile 的所有伙伴共用此设置。层叠层数不超过保留数量;阅读时的新消息也按保留数量排队。', 'companion.capacityError': '层数须为 1–10,保留数量须为 1–100 的整数。', @@ -2881,6 +2884,9 @@ export const en = { 'companion.scope.shared': 'Channels shared with me', 'companion.scope.all-bot': 'All channels the Bot joins', 'companion.settings': 'Window Companions', + 'companion.speechSound': 'Companion sounds', + 'companion.speechSoundHint': + 'Soft babble follows new message text, with sounds for grabbing, fast dragging, throwing and landing. No spoken narration. Off by default; applies to all Window Companions in this Profile.', 'companion.capacityHint': 'Shared by all companions in this Profile. Layers stay within retention; arrivals while reading use a queue bounded by retention too.', 'companion.capacityError': 'Use integers: 1–10 layers and 1–100 retained cards.', diff --git a/packages/client/src/client/window-companion-styles.ts b/packages/client/src/client/window-companion-styles.ts index 921888ba72..9401b168cb 100644 --- a/packages/client/src/client/window-companion-styles.ts +++ b/packages/client/src/client/window-companion-styles.ts @@ -9,7 +9,8 @@ export const WINDOW_COMPANION_CSS = ` --bh-companion-attention: var(--dsw-alias-state-warn-primary); --bh-companion-font: var(--dsw-font-markdown-code-block-font-family); /* @bh-companion-aliases:end */ - position: absolute; inset: 0; pointer-events: none; overflow: hidden; + /* Focus may scroll hidden overflow; the presentation viewport must stay fixed. */ + position: absolute; inset: 0; pointer-events: none; overflow: clip; } .bh-companion-chip { display: inline-flex; align-items: center; gap: 2px; } .bh-channel-island-wrap > .bh-companion-pin { margin-left: 2px; } @@ -41,7 +42,8 @@ export const WINDOW_COMPANION_CSS = ` .bh-companion-pending .bh-tool-approval-card { margin: 0; min-width: 0; grid-template-columns: minmax(0, 1fr); overflow-wrap: anywhere; } .bh-companion-pending .bh-tool-approval-input { max-height: 160px; overflow: auto; white-space: pre-wrap; overflow-wrap: anywhere; } .bh-companion-cards { position: absolute; bottom: 174px; width: 320px; max-width: calc(100vw - 16px); margin: 0; padding: 0; list-style: none; overflow: visible; transition: height 220ms ease; } -.bh-companion[data-reading='true'] .bh-companion-cards { overflow: auto; scrollbar-gutter: stable; } +.bh-companion[data-reading='true'] .bh-companion-cards { overflow: auto; scrollbar-width: none; } +.bh-companion-cards::-webkit-scrollbar, .bh-companion-card p::-webkit-scrollbar { display: none; } .bh-companion-card { position: absolute; width: 100%; height: 104px; padding: 8px 10px; background: var(--bh-companion-surface); border: 2px solid var(--bh-companion-border); transform-origin: center top; color: var(--bh-companion-text); transition: top 220ms ease, transform 220ms ease; box-shadow: 3px 3px 0 var(--bh-companion-border); } .bh-companion-card header { display: flex; align-items: center; justify-content: space-between; gap: 8px; } .bh-companion-source { display: flex; flex: 1; flex-direction: column; min-width: 0; gap: 2px; } @@ -49,12 +51,16 @@ export const WINDOW_COMPANION_CSS = ` .bh-companion-source button { text-align: left; line-height: 14px; } .bh-companion-source button:disabled { cursor: default; } .bh-companion-card header button { font-size: 11px; color: var(--bh-companion-muted); background: transparent; border: 0; padding: 0; cursor: pointer; overflow: hidden; white-space: nowrap; text-overflow: ellipsis; } -.bh-companion-card p { margin: 6px 0 0; font: 12px/18px var(--bh-companion-font); white-space: pre-wrap; overflow-wrap: anywhere; max-height: 54px; overflow: auto; } +.bh-companion-card p { margin: 6px 0 0; font: 12px/18px var(--bh-companion-font); white-space: pre-wrap; overflow-wrap: anywhere; max-height: 54px; overflow-x: hidden; overflow-y: auto; scrollbar-width: none; overflow-anchor: none; scroll-behavior: auto; } +.bh-companion-card p:focus-visible { outline: 2px solid var(--bh-accent); outline-offset: 2px; } .bh-companion-card p[data-context='true'] { max-height: 36px; } -.bh-companion-toolbar { position: absolute; bottom: 96px; left: -10px; display: flex; align-items: center; gap: 4px; padding: 5px; background: var(--bh-companion-surface); border: 1px solid var(--bh-companion-border); opacity: 0; visibility: hidden; transition: opacity 150ms ease; } -.bh-companion-toolbar[data-open='true'] { opacity: 1; visibility: visible; } +.bh-companion-toolbar { position: absolute; bottom: 96px; left: -10px; display: flex; align-items: center; gap: 4px; padding: 5px; background: var(--bh-companion-surface); border: 1px solid var(--bh-companion-border); } .bh-companion-toolbar button { display: grid; place-items: center; min-width: 28px; height: 26px; border: 0; background: transparent; color: var(--bh-companion-text); cursor: pointer; } .bh-companion-toolbar button:hover { background: var(--bh-companion-hover); } .bh-companion-attention { position: absolute; z-index: 2; right: 4px; top: 4px; min-width: 20px; height: 20px; padding: 0 3px; border: 1px solid var(--bh-companion-border); background: var(--bh-companion-surface); color: var(--bh-companion-attention); cursor: pointer; } +.bh-companion-toolbar, .bh-companion-attention { opacity: 0; visibility: hidden; pointer-events: none; transition: opacity 150ms ease, visibility 0s linear 150ms; } +.bh-companion:hover .bh-companion-toolbar, .bh-companion:has(:focus-visible) .bh-companion-toolbar, +.bh-companion:hover .bh-companion-attention, .bh-companion:has(:focus-visible) .bh-companion-attention, +.bh-companion-toolbar[data-open='true'] { opacity: 1; visibility: visible; pointer-events: auto; transition-delay: 0s; } html[data-botharness-motion='reduce'] .bh-companion * { transition: none; } `; diff --git a/packages/client/src/client/window-companion-view.tsx b/packages/client/src/client/window-companion-view.tsx index 216f9e7516..a82a95606e 100644 --- a/packages/client/src/client/window-companion-view.tsx +++ b/packages/client/src/client/window-companion-view.tsx @@ -23,6 +23,7 @@ import { CompanionMotion, type CompanionPoint } from './companion-motion.js'; import { CompanionBubbles, type BubblePlacement } from './companion-bubbles.js'; import { isAvatarAppearance } from '../../../core/src/bots/avatar-appearance.js'; import { companionMessageIdentity } from '../../../core/src/companions/sources.js'; +import { companionBabble, type CompanionSound } from './companion-sound.js'; import type { BridgeActions } from './actions.js'; import { CompanionRequests } from './companion-requests-view.js'; import type { AvatarAnchor } from './avatar-anchor.js'; @@ -36,12 +37,35 @@ function avatarLimitation( } const statusVisible = (state: CompanionViewState): boolean => Boolean( - state.selection?.activity || + (state.selection?.activity && state.activity && state.activity.state !== 'idle') || state.bot?.paused || avatarLimitation(state.bot) || state.sync !== 'live', ); +function CompanionMessageText({ text, context }: { text: string; context: boolean }): ReactElement { + const following = useRef(true); + const viewport = useMountedResource( + (node) => { + if (following.current) node.scrollTop = Math.max(0, node.scrollHeight - node.clientHeight); + }, + [text, context], + ); + return ( +

{ + const node = event.currentTarget; + following.current = node.scrollHeight - node.clientHeight - node.scrollTop <= 1; + }} + > + {text} +

+ ); +} + export interface WindowCompanionViewProps { companion: WindowCompanion; openDm(botId: string): void; @@ -51,6 +75,7 @@ export interface WindowCompanionViewProps { onRemove?(botId: string): void; bubbles?: CompanionBubbles | undefined; openSettings?(): void; + sound?: CompanionSound | undefined; actions?: BridgeActions | undefined; } export function WindowCompanionView({ @@ -62,13 +87,34 @@ export function WindowCompanionView({ onRemove, bubbles, openSettings, + sound, actions, }: WindowCompanionViewProps): ReactElement | null { const view = useSyncExternalStore(companion.subscribe, companion.getSnapshot); const latest = useRef(view); latest.current = view; - const [motion] = useState(() => new CompanionMotion()); + const audio = useRef(sound); + audio.current = sound; + const audible = useRef(false); + const [motion] = useState( + () => + new CompanionMotion((event) => { + const state = latest.current; + if ( + audible.current && + !document.hidden && + document.documentElement.dataset['botharnessMotion'] !== 'reduce' && + state.selection && + state.bot && + !state.bot.paused && + state.sync === 'live' + ) + audio.current?.interact(state.selection.botId, event); + }), + ); const [point, setPoint] = useState(motion.point); + const presentPoint = useRef<(next: CompanionPoint) => void>(setPoint); + const motionFrameAt = useRef(performance.now()); const displayedPoint = useRef(point); displayedPoint.current = point; const anchor = useRef(undefined); @@ -98,6 +144,7 @@ export function WindowCompanionView({ const hovering = useRef(false); const exit = useRef | undefined>(undefined); const clickReset = useRef | undefined>(undefined); + const suppressClick = useRef(false); const [menu, setMenu] = useState(false); const menuClass = `bh-companion-menu-${useId()}`; const menuTrigger = useRef(null); @@ -119,11 +166,16 @@ export function WindowCompanionView({ const reducedMotion = (): boolean => document.documentElement.dataset['botharnessMotion'] === 'reduce'; const persistPosition = (): void => companion.configure({ position: motion.position() }); + const releaseMotion = (cancelled = false): void => { + const now = performance.now(); + motionFrameAt.current = now; + presentPoint.current(motion.release(now, reducedMotion(), cancelled)); + }; const cancelDrag = (event: { pointerId: number }): void => { if (!pointer.current || pointer.current.id !== event.pointerId || motion.point.phase !== 'drag') return; pointer.current = undefined; - setPoint(motion.release(performance.now(), reducedMotion(), true)); + releaseMotion(true); persistPosition(); }; const leave = (): void => { @@ -131,7 +183,12 @@ export function WindowCompanionView({ if (exit.current !== undefined) clearTimeout(exit.current); exit.current = setTimeout(() => { exit.current = undefined; - if (root.current?.contains(document.activeElement) || hovering.current || menuOpen.current) + if ( + (root.current?.contains(document.activeElement) && + document.activeElement?.matches(':focus-visible')) || + hovering.current || + menuOpen.current + ) return; companion.reading(false); }, 800); @@ -164,6 +221,14 @@ export function WindowCompanionView({ let elapsed = 0; let measured = false; let visible = true; + let stageVisible = true; + const syncAudio = () => { + const state = companion.getSnapshot(); + if (!state.selection || !state.bot || state.bot.paused || state.sync !== 'live') + sound?.stop(state.selection?.botId ?? view.selection?.botId); + else if (!state.cards.length) sound?.stopSpeech(state.selection.botId); + }; + const unsubscribeAudio = companion.subscribe(syncAudio); const placeBubble = (next: CompanionPoint, sampled: ReturnType) => { const state = latest.current; if (state.selection) { @@ -202,6 +267,10 @@ export function WindowCompanionView({ } } }; + presentPoint.current = (next) => { + placeBubble(next, anchor.current?.read()); + setPoint(next); + }; const measure = () => { const box = node.getBoundingClientRect(); const width = box.width; @@ -225,23 +294,31 @@ export function WindowCompanionView({ drag.originX = next.x; drag.originY = next.y; } - setPoint(next); - placeBubble(next, anchor.current?.read()); + presentPoint.current(next); }; const tick = (now: number) => { frame = 0; - const milliseconds = Math.min(100, Math.max(0, now - previous)); + const milliseconds = Math.max(0, now - previous); + const motionMilliseconds = Math.max(0, now - motionFrameAt.current); + motionFrameAt.current = now; previous = now; const reduced = reducedMotion(); const state = latest.current; - if (!document.hidden && visible) { - elapsed += milliseconds; + if (!document.hidden && stageVisible) { + if (visible) elapsed += Math.min(100, milliseconds); if (elapsed >= 50) { + const before = companion.getSnapshot().cards; companion.advance(elapsed, reduced); + const after = companion.getSnapshot(); + if (!reduced && after.sync === 'live' && after.bot && !after.bot.paused) { + const text = companionBabble(before, after.cards); + if (text) sound?.play(after.bot.slug, text); + } elapsed = 0; } const walking = Boolean( state.selection?.walking && + visible && state.bot?.paused !== true && state.sync === 'live' && !state.reading && @@ -254,33 +331,44 @@ export function WindowCompanionView({ statusVisible(state) || state.cards.length || state.requests.length ? anchor.current?.read() : undefined; - const next = motion.advance(milliseconds, reduced, walking, direction.current); + const next = motion.advance(motionMilliseconds, reduced, walking, direction.current); placeBubble(next, sampled); if (walking && (next.x <= 8 || next.x >= Math.max(8, next.width - 104))) direction.current *= -1; if (next !== previousPoint) setPoint(next); if (previousPoint.phase !== 'rest' && next.phase === 'rest') persistPosition(); } - if (!document.hidden && visible) frame = requestAnimationFrame(tick); + if (!document.hidden && stageVisible) frame = requestAnimationFrame(tick); }; const visibility = () => { + audible.current = !document.hidden && stageVisible && visible; + if (document.hidden || !stageVisible || !visible) + sound?.stop(latest.current.selection?.botId); cancelAnimationFrame(frame); frame = 0; previous = performance.now(); + motionFrameAt.current = previous; elapsed = 0; - if (!document.hidden && visible) frame = requestAnimationFrame(tick); + if (!document.hidden && stageVisible) frame = requestAnimationFrame(tick); }; const observer = typeof IntersectionObserver === 'undefined' ? undefined : new IntersectionObserver((entries) => { - visible = entries.some((entry) => entry.isIntersecting); + for (const entry of entries) { + if (entry.target === node) stageVisible = entry.isIntersecting; + else visible = entry.isIntersecting; + } visibility(); }); + observer?.observe(node); const target = node.querySelector('.bh-companion'); if (target) observer?.observe(target); const policy = new MutationObserver(() => { - if (reducedMotion()) setPoint(motion.advance(0, true, false, direction.current)); + if (reducedMotion()) { + sound?.stop(latest.current.selection?.botId); + presentPoint.current(motion.advance(0, true, false, direction.current)); + } }); policy.observe(document.documentElement, { attributes: true, @@ -291,6 +379,9 @@ export function WindowCompanionView({ window.addEventListener('resize', measure); document.addEventListener('visibilitychange', visibility); return () => { + audible.current = false; + unsubscribeAudio(); + sound?.stop(latest.current.selection?.botId ?? view.selection?.botId); cancelAnimationFrame(frame); observer?.disconnect(); policy.disconnect(); @@ -299,11 +390,12 @@ export function WindowCompanionView({ if (exit.current !== undefined) clearTimeout(exit.current); if (clickReset.current !== undefined) clearTimeout(clickReset.current); pointer.current = undefined; + presentPoint.current = setPoint; if (latest.current.selection) bubbleOwner.remove(latest.current.selection.botId); root.current = null; }; }, - [companion, view.selection?.botId, bubbleOwner], + [companion, view.selection?.botId, bubbleOwner, sound], ); if (!view.selection || !view.bot) return null; const { selection, bot, activity } = view; @@ -372,7 +464,12 @@ export function WindowCompanionView({ data-sync={view.sync} data-motion={point.phase} data-bot={bot.slug} - style={{ left: point.x, bottom: point.y, zIndex: view.reading ? 10 : 1 }} + style={{ + left: 0, + bottom: 0, + transform: `translate3d(${point.x}px, ${-point.y}px, 0)`, + zIndex: view.reading ? 10 : 1, + }} onKeyDown={(event) => { if (event.key === 'Escape' && menu) { event.preventDefault(); @@ -387,7 +484,9 @@ export function WindowCompanionView({ }} onPointerEnter={enter} onPointerLeave={leave} - onFocusCapture={enter} + onFocusCapture={(event) => { + if (event.target.matches(':focus-visible')) enter(); + }} onBlurCapture={(event) => { if (!event.currentTarget.contains(event.relatedTarget)) leave(); }} @@ -525,13 +624,16 @@ export function WindowCompanionView({ -

{card.body.slice(0, card.shown)}

+ ); })} ) : null} -
+
+ <> +
+
+
{t('companion.speechSound')}
+
{t('companion.speechSoundHint')}
+
+ companion.configureSpeechSound(enabled)} + disabled={!state.ready} + label={t('companion.speechSound')} + />
- +
{ + event.preventDefault(); + const data = new FormData(event.currentTarget); + const layers = Number(data.get('layers')); + const retention = Number(data.get('retention')); + if ( + !Number.isInteger(layers) || + layers < 1 || + layers > 10 || + !Number.isInteger(retention) || + retention < 1 || + retention > 100 + ) { + setError(true); + return; + } + companion.configureCapacity({ layers, retention }); + setError(false); + }} + > +
+
{t('companion.settings')}
+
{t('companion.capacityHint')}
+ {error ?
{t('companion.capacityError')}
: null} +
+
+ + + +
+
+ ); } diff --git a/packages/client/src/client/window-companions.ts b/packages/client/src/client/window-companions.ts index 5d08671cad..91ba7ede6c 100644 --- a/packages/client/src/client/window-companions.ts +++ b/packages/client/src/client/window-companions.ts @@ -93,8 +93,15 @@ export class WindowCompanions { ready: boolean; companions: readonly WindowCompanion[]; capacity: CompanionCapacity; + speechSound: boolean; sync: 'connecting' | 'live' | 'stale'; - } = { ready: false, companions: [], capacity: { layers: 3, retention: 20 }, sync: 'connecting' }; + } = { + ready: false, + companions: [], + capacity: { layers: 3, retention: 20 }, + speechSound: false, + sync: 'connecting', + }; private listeners = new Set<() => void>(); private children = new Map(); private preferences = new Map(); @@ -147,7 +154,11 @@ export class WindowCompanions { this.hadSavedPreferences = raw !== null && raw !== undefined; if (data) { selections = Array.isArray(data['selections']) ? data['selections'] : []; - this.state = { ...this.state, capacity: capacity(data['capacity']) }; + this.state = { + ...this.state, + capacity: capacity(data['capacity']), + speechSound: data['speechSound'] === true, + }; } else { const legacy = this.deps.storage?.getItem(`botharness/companions/v1/${profileId}`); this.hadSavedPreferences ||= Boolean(legacy); @@ -238,6 +249,12 @@ export class WindowCompanions { this.save(); this.notify(); } + configureSpeechSound(enabled: boolean): void { + if (!this.state.ready || this.disposed || this.state.speechSound === enabled) return; + this.state = { ...this.state, speechSound: enabled }; + this.save(); + this.notify(); + } private get key(): string { return `botharness/companions/v2/${this.profileId}`; } @@ -249,6 +266,7 @@ export class WindowCompanions { JSON.stringify({ selections: [...this.preferences.values()], capacity: this.state.capacity, + speechSound: this.state.speechSound, }), ); } catch {} diff --git a/packages/client/test/companion-motion.test.ts b/packages/client/test/companion-motion.test.ts new file mode 100644 index 0000000000..ebaddb1d95 --- /dev/null +++ b/packages/client/test/companion-motion.test.ts @@ -0,0 +1,99 @@ +import { expect, it } from 'vitest'; +import { CompanionMotion } from '../src/client/companion-motion.js'; + +it('reports a real grab, cooled fast drag, fast throw and one first-ground impact without bounce chatter', () => { + const feedback: { kind: string; strength: number }[] = []; + const motion = new CompanionMotion((event) => feedback.push(event)); + motion.measure(1200, 800, 0.4); + motion.grab(0, false); + expect(feedback).toEqual([]); + motion.drag(450, 200, 16, false); + motion.drag(466, 200, 32, false); + motion.drag(482, 200, 48, false); + motion.release(64, false); + expect(feedback.map((event) => event.kind)).toEqual(['grab', 'drag', 'throw']); + for (let frame = 0; frame < 300; frame++) motion.advance(16, false, false, 1); + expect(feedback.map((event) => event.kind)).toEqual(['grab', 'drag', 'throw', 'land']); + expect(feedback.at(-1)!.strength).toBeGreaterThan(0); + expect(feedback.at(-1)!.strength).toBeLessThanOrEqual(1); + expect(motion.point).toMatchObject({ phase: 'rest', y: 0 }); +}); + +it('keeps clicks, cancelled drops, held releases and reduced-motion resets free of stale interaction sounds', () => { + const feedback: string[] = []; + const motion = new CompanionMotion((event) => feedback.push(event.kind)); + motion.measure(1200, 800, 0.4); + motion.grab(0, false); + motion.release(16, false, true); + for (let frame = 0; frame < 200; frame++) motion.advance(16, false, false, 1); + expect(feedback).toEqual([]); + motion.grab(4000, false); + motion.drag(450, 200, 4016, false); + motion.drag(466, 200, 4032, false); + feedback.length = 0; + motion.release(4048, false, true); + for (let frame = 0; frame < 200; frame++) motion.advance(16, false, false, 1); + expect(feedback).toEqual([]); + motion.grab(8000, false); + motion.drag(450, 200, 8016, false); + motion.drag(466, 200, 8032, false); + feedback.length = 0; + motion.release(9000, false); + expect(feedback).toEqual([]); + motion.advance(0, true, false, 1); + for (let frame = 0; frame < 200; frame++) motion.advance(16, false, false, 1); + expect(feedback).toEqual([]); + motion.grab(12000, true); + motion.drag(500, 200, 12016, true); + motion.drag(700, 300, 12032, true); + motion.release(12048, true); + expect(feedback).toEqual([]); +}); + +it('keeps angular inertia through a drag reversal, settles while held, and retains bottom alignment on release', () => { + const motion = new CompanionMotion(); + motion.measure(1200, 800, 0.4); + motion.grab(0, false); + for (let i = 1; i <= 12; i++) { + motion.drag(motion.point.x + 5, 250, i * 16, false); + motion.advance(16, false, false, 1); + } + expect(motion.point.tilt).toBeGreaterThan(1); + const angle = motion.point.tilt; + motion.drag(motion.point.x - 8, 250, 208, false); + expect(motion.point.tilt).toBe(angle); + motion.advance(8, false, false, 1); + expect(Math.abs(motion.point.tilt - angle)).toBeLessThan(2); + for (let i = 1; i <= 30; i++) { + motion.drag(motion.point.x - 5, 250, 208 + i * 16, false); + motion.advance(16, false, false, 1); + } + expect(motion.point.tilt).toBeLessThan(-1); + const trace: number[] = []; + for (let i = 0; i < 200; i++) trace.push(motion.advance(16, false, false, 1).tilt); + expect(trace.some((angle) => angle > 0)).toBe(true); + expect(trace.every((angle) => Math.abs(angle) <= 24)).toBe(true); + expect(motion.point.tilt).toBe(0); + motion.release(4000, false); + for (let i = 0; i < 250; i++) motion.advance(16, false, false, 1); + expect(motion.point).toMatchObject({ y: 0, tilt: 0, squash: 0, phase: 'rest' }); +}); + +it('bounds a delayed frame and clears spring energy when motion is reduced or a position is reset', () => { + const motion = new CompanionMotion(); + motion.measure(1000, 700, 0.4); + motion.grab(0, false); + motion.drag(600, 200, 16, false); + const delayed = motion.advance(60000, false, false, 1); + expect(Number.isFinite(delayed.tilt)).toBe(true); + expect(Math.abs(delayed.tilt)).toBeLessThanOrEqual(24); + expect(motion.advance(0, true, false, 1)).toMatchObject({ tilt: 0, squash: 0, phase: 'drag' }); + motion.release(32, true); + expect(motion.point).toMatchObject({ y: 0, tilt: 0, squash: 0, phase: 'rest' }); + expect(motion.advance(16, false, false, 1).tilt).toBe(0); + motion.grab(50, false); + motion.drag(400, 200, 66, false); + motion.advance(64, false, false, 1); + motion.move(200); + expect(motion.advance(64, false, false, 1)).toMatchObject({ x: 200, tilt: 0, phase: 'rest' }); +}); diff --git a/packages/client/test/companion-position.ts b/packages/client/test/companion-position.ts new file mode 100644 index 0000000000..7925ceddfc --- /dev/null +++ b/packages/client/test/companion-position.ts @@ -0,0 +1,7 @@ +import { expect } from 'vitest'; + +export function companionPosition(surface: HTMLElement): { left: string; bottom: string } { + const match = /^translate3d\(([-\d.]+)px, ([-\d.]+)px, 0\)$/.exec(surface.style.transform); + expect(match).not.toBeNull(); + return { left: `${Number(match![1])}px`, bottom: `${-Number(match![2])}px` }; +} diff --git a/packages/client/test/companion-sound.test.ts b/packages/client/test/companion-sound.test.ts new file mode 100644 index 0000000000..24a0a6200d --- /dev/null +++ b/packages/client/test/companion-sound.test.ts @@ -0,0 +1,267 @@ +import { afterEach, expect, it, vi } from 'vitest'; +import { CompanionSound, companionBabble } from '../src/client/companion-sound.js'; +import { WindowCompanion } from '../src/client/window-companion.js'; + +class AudioParameter { + setValueAtTime = vi.fn(); + exponentialRampToValueAtTime = vi.fn(); +} +class Oscillator { + type = 'sine'; + frequency = new AudioParameter(); + onended: (() => void) | null = null; + connect = vi.fn(); + disconnect = vi.fn(); + start = vi.fn(); + stop = vi.fn(); +} +class Gain { + gain = new AudioParameter(); + connect = vi.fn(); + disconnect = vi.fn(); +} +class NoiseSource extends Oscillator { + buffer: unknown; +} +class Filter extends Oscillator { + Q = new AudioParameter(); +} +class Audio { + static instances: Audio[] = []; + state = 'suspended'; + currentTime = 0; + destination = {}; + oscillators: Oscillator[] = []; + gains: Gain[] = []; + sampleRate = 48000; + buffers: { getChannelData: () => Float32Array }[] = []; + noises: NoiseSource[] = []; + filters: Filter[] = []; + resume = vi.fn(async () => { + this.state = 'running'; + }); + close = vi.fn(async () => { + this.state = 'closed'; + }); + constructor() { + Audio.instances.push(this); + } + createOscillator() { + const oscillator = new Oscillator(); + this.oscillators.push(oscillator); + return oscillator; + } + createGain() { + const gain = new Gain(); + this.gains.push(gain); + return gain; + } + createBuffer(_channels: number, length: number) { + const samples = new Float32Array(length); + const buffer = { getChannelData: () => samples }; + this.buffers.push(buffer); + return buffer; + } + createBufferSource() { + const source = new NoiseSource(); + this.noises.push(source); + return source; + } + createBiquadFilter() { + const filter = new Filter(); + this.filters.push(filter); + return filter; + } +} + +afterEach(() => { + Audio.instances = []; + vi.unstubAllGlobals(); +}); + +it('reuses a bounded throw-noise buffer and releases its filter after completion or partial allocation failure', () => { + vi.stubGlobal('AudioContext', Audio); + const sound = new CompanionSound(); + sound.setEnabled(true); + sound.unlock(); + const audio = Audio.instances[0]!; + sound.interact('ada', { kind: 'throw', strength: 1 }); + expect(audio.buffers).toHaveLength(1); + expect(audio.buffers[0]!.getChannelData().length).toBeLessThan(audio.sampleRate); + expect(audio.noises[0]!.buffer).toBe(audio.buffers[0]); + expect(audio.filters[0]!.type).toBe('bandpass'); + audio.noises[0]!.onended!(); + expect(audio.noises[0]!.disconnect).toHaveBeenCalledOnce(); + expect(audio.filters[0]!.disconnect).toHaveBeenCalledOnce(); + vi.spyOn(audio, 'createGain').mockImplementationOnce(() => { + throw new Error('Unavailable'); + }); + expect(() => sound.interact('ada', { kind: 'throw', strength: 1 })).not.toThrow(); + expect(audio.noises[1]!.disconnect).toHaveBeenCalledOnce(); + expect(audio.filters[1]!.disconnect).toHaveBeenCalledOnce(); + sound.interact('ada', { kind: 'throw', strength: 1 }); + expect(audio.buffers).toHaveLength(1); + expect(audio.noises[2]!.buffer).toBe(audio.buffers[0]); + sound.interact('ada', { kind: 'grab', strength: 1 }); + expect(audio.noises[2]!.disconnect).toHaveBeenCalledOnce(); + sound.interact('ada', { kind: 'throw', strength: 1 }); + expect(audio.oscillators.at(-1)!.disconnect).toHaveBeenCalledOnce(); + expect(audio.noises).toHaveLength(4); + sound.dispose(); + expect(audio.noises[3]!.disconnect).toHaveBeenCalledOnce(); + expect(audio.filters[3]!.disconnect).toHaveBeenCalledOnce(); +}); + +it('shares opt-in, gesture admission and bounded resources between speech and interaction sounds', () => { + vi.stubGlobal('AudioContext', Audio); + const sound = new CompanionSound(); + sound.interact('ada', { kind: 'grab', strength: 1 }); + sound.setEnabled(true); + sound.interact('ada', { kind: 'land', strength: 1 }); + expect(Audio.instances).toHaveLength(0); + sound.unlock(); + const audio = Audio.instances[0]!; + sound.play('ada', 'a'); + sound.interact('ada', { kind: 'grab', strength: 1 }); + expect(audio.oscillators).toHaveLength(2); + expect(audio.oscillators[0]!.disconnect).toHaveBeenCalledOnce(); + expect(audio.oscillators[1]!.type).toBe('triangle'); + sound.stopSpeech('ada'); + expect(audio.oscillators[1]!.disconnect).not.toHaveBeenCalled(); + sound.interact('grace', { kind: 'land', strength: 0.3 }); + sound.interact('third', { kind: 'grab', strength: 1 }); + expect(audio.oscillators).toHaveLength(3); + sound.setEnabled(false); + expect(audio.oscillators[1]!.disconnect).toHaveBeenCalledOnce(); + expect(audio.oscillators[2]!.disconnect).toHaveBeenCalledOnce(); + sound.setEnabled(true); + expect(audio.oscillators).toHaveLength(3); + sound.dispose(); + expect(audio.close).toHaveBeenCalledOnce(); +}); + +it('requires opt-in and a gesture, drops blocked playback, and never queues it for a later unlock', async () => { + vi.stubGlobal('AudioContext', Audio); + const sound = new CompanionSound(); + sound.unlock(); + sound.play('ada', 'a'); + expect(Audio.instances).toHaveLength(0); + sound.setEnabled(true); + sound.play('ada', 'a'); + expect(Audio.instances).toHaveLength(0); + sound.unlock(); + const audio = Audio.instances[0]!; + audio.state = 'suspended'; + audio.resume.mockRejectedValue(new Error('Autoplay blocked')); + sound.unlock(); + sound.play('ada', 'a'); + await Promise.resolve(); + expect(audio.oscillators).toHaveLength(0); + audio.state = 'running'; + sound.unlock(); + expect(audio.oscillators).toHaveLength(0); + sound.play('ada', 'b'); + expect(audio.oscillators).toHaveLength(1); + sound.setEnabled(false); + expect(audio.oscillators[0]!.stop).toHaveBeenLastCalledWith(); + expect(audio.oscillators[0]!.disconnect).toHaveBeenCalledOnce(); + expect(audio.gains[0]!.disconnect).toHaveBeenCalledOnce(); + sound.setEnabled(true); + expect(audio.oscillators).toHaveLength(1); + sound.dispose(); + expect(audio.close).toHaveBeenCalledOnce(); +}); + +it('bounds concurrent and per-Bot sound resources, varies syllables, and cancels only the removed Bot', () => { + vi.stubGlobal('AudioContext', Audio); + const sound = new CompanionSound(); + sound.setEnabled(true); + sound.unlock(); + const audio = Audio.instances[0]!; + sound.play('ada', 'a'); + audio.currentTime = 0.02; + sound.play('grace', 'g'); + expect(audio.oscillators).toHaveLength(1); + audio.currentTime = 0.04; + sound.play('ada', 'b'); + expect(audio.oscillators).toHaveLength(1); + sound.play('grace', 'g'); + expect(audio.oscillators).toHaveLength(2); + audio.currentTime = 0.1; + sound.play('third', 'c'); + expect(audio.oscillators).toHaveLength(2); + sound.stop('ada'); + expect(audio.oscillators[0]!.onended).toBeNull(); + expect(audio.oscillators[1]!.disconnect).not.toHaveBeenCalled(); + sound.play('third', 'c'); + expect(audio.oscillators).toHaveLength(3); + expect(audio.oscillators[0]!.frequency.setValueAtTime.mock.calls[0]![0]).not.toBe( + audio.oscillators[1]!.frequency.setValueAtTime.mock.calls[0]![0], + ); + audio.oscillators[1]!.onended!(); + expect(audio.gains[1]!.disconnect).toHaveBeenCalledOnce(); + sound.dispose(); + expect(audio.oscillators[2]!.disconnect).toHaveBeenCalledOnce(); + expect(audio.close).toHaveBeenCalledOnce(); +}); + +it('silently degrades without audio support and releases partial allocations after a synthesis failure', () => { + const sound = new CompanionSound(); + sound.setEnabled(true); + vi.stubGlobal('AudioContext', undefined); + expect(() => sound.unlock()).not.toThrow(); + expect(() => sound.play('ada', 'a')).not.toThrow(); + vi.stubGlobal('AudioContext', Audio); + sound.unlock(); + const audio = Audio.instances[0]!; + vi.spyOn(audio, 'createGain').mockImplementationOnce(() => { + throw new Error('Resource unavailable'); + }); + expect(() => sound.play('ada', 'a')).not.toThrow(); + expect(audio.oscillators[0]!.stop).toHaveBeenCalledOnce(); + expect(audio.oscillators[0]!.disconnect).toHaveBeenCalledOnce(); + sound.play('ada', 'b'); + expect(audio.oscillators).toHaveLength(2); + sound.dispose(); +}); + +it('admits only newly revealed committed graphemes, with silent punctuation and no historical catch-up', async () => { + const events = new EventTarget(); + const owner = new WindowCompanion({ + context: async () => ({ profileId: 'qa' }), + source: () => ({ addEventListener: events.addEventListener.bind(events), close() {} }), + }); + const send = (name: string, data: unknown) => + events.dispatchEvent(new MessageEvent(name, { data: JSON.stringify(data) })); + await owner.start(); + owner.select('ada'); + send('companion/baseline', { + profileId: 'qa', + bot: { slug: 'ada', name: 'Ada', paused: false }, + activity: { generation: 'host', revision: 0, bots: [] }, + }); + send('companion/message', { + generation: 'host', + botId: 'ada', + channelId: 'dm', + channelName: 'Ada', + messageId: 'fresh', + body: '你,👩🏽‍💻 A', + source: 'own-dm', + }); + const reveal = () => { + const before = owner.getSnapshot().cards; + owner.advance(35); + return companionBabble(before, owner.getSnapshot().cards); + }; + expect(reveal()).toBe('你'); + expect(reveal()).toBeUndefined(); + expect(reveal()).toBe('👩🏽‍💻'); + expect(reveal()).toBeUndefined(); + expect(reveal()).toBe('A'); + const shown = owner.getSnapshot().cards; + expect(companionBabble([], shown)).toBeUndefined(); + expect(companionBabble(shown, shown)).toBeUndefined(); + expect(reveal()).toBeUndefined(); + owner.dispose(); +}); diff --git a/packages/client/test/window-companion-view.test.ts b/packages/client/test/window-companion-view.test.ts index 75024e228b..752e53b953 100644 --- a/packages/client/test/window-companion-view.test.ts +++ b/packages/client/test/window-companion-view.test.ts @@ -5,6 +5,8 @@ import { expect, it, vi } from 'vitest'; import { WindowCompanion } from '../src/client/window-companion.js'; import { WindowCompanionView } from '../src/client/window-companion-view.js'; import { zhTranslate } from '../src/client/locale.js'; +import { CompanionSound } from '../src/client/companion-sound.js'; +import { companionPosition } from './companion-position.js'; vi.mock('@deepseek-ai/dsh-client-ui-primitives', () => ({ Menu: ({ @@ -39,7 +41,203 @@ vi.mock('@deepseek-ai/dsh-client-ui-primitives', () => ({ IconNewChatOutlineRegular: () => null, })); -it('projects the owning Avatar screen point into an offset overlay and remeasures it after resize', async () => { +it('follows revealed message tails independently and preserves upward reading until returning to the bottom', async () => { + Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + vi.stubGlobal('requestAnimationFrame', () => 1); + vi.stubGlobal('cancelAnimationFrame', () => {}); + const events = new EventTarget(); + const companion = new WindowCompanion({ + context: async () => ({ profileId: 'qa' }), + source: () => ({ addEventListener: events.addEventListener.bind(events), close() {} }), + }); + const send = (type: string, value: unknown) => + events.dispatchEvent(new MessageEvent(type, { data: JSON.stringify(value) })); + await companion.start(); + companion.select('ada'); + send('companion/baseline', { + profileId: 'qa', + bot: { slug: 'ada', name: 'Ada', paused: false }, + activity: { generation: 'host', revision: 0, bots: [] }, + }); + const node = document.createElement('div'); + document.body.append(node); + const root = createRoot(node); + let firstHeight = 180; + let secondHeight = 90; + const height = vi + .spyOn(HTMLElement.prototype, 'scrollHeight', 'get') + .mockImplementation(function (this: HTMLElement) { + if (!this.matches('.bh-companion-card p') || !this.textContent) return 0; + return this.textContent.startsWith('A') ? firstHeight : secondHeight; + }); + const viewport = vi.spyOn(HTMLElement.prototype, 'clientHeight', 'get').mockReturnValue(54); + try { + await act(() => + root.render( + createElement(WindowCompanionView, { + companion, + openDm() {}, + openAttention() {}, + openChannel() {}, + t: zhTranslate, + }), + ), + ); + await act(() => { + for (const letter of ['A', 'B']) + send('companion/message', { + generation: 'host', + botId: 'ada', + channelId: 'dm', + channelName: 'Ada', + messageId: letter, + body: letter.repeat(200), + source: 'own-dm', + }); + companion.advance(350); + }); + const [first, second] = [ + ...node.querySelectorAll('.bh-companion-card p'), + ]; + expect(first!.scrollTop).toBe(126); + expect(second!.scrollTop).toBe(36); + await act(() => { + first!.scrollTop = 18; + first!.dispatchEvent(new Event('scroll')); + }); + firstHeight = 216; + secondHeight = 126; + await act(() => companion.advance(350)); + expect(first!.scrollTop).toBe(18); + expect(second!.scrollTop).toBe(72); + expect(first!.textContent).toBe('A'.repeat(20)); + await act(() => { + first!.scrollTop = 162; + first!.dispatchEvent(new Event('scroll')); + }); + firstHeight = 252; + await act(() => companion.advance(350)); + expect(first!.scrollTop).toBe(198); + await act(() => companion.advance(30_000, true)); + expect(first!.textContent).toBe('A'.repeat(200)); + expect(second!.textContent).toBe('B'.repeat(200)); + expect(node.querySelectorAll('.bh-companion-card p')[0]).toBe(first); + } finally { + await act(() => root.unmount()); + companion.dispose(); + node.remove(); + height.mockRestore(); + viewport.mockRestore(); + vi.unstubAllGlobals(); + } +}); + +it('sounds only fresh playback and silences a Bot on background, stale sync, archive and unmount', async () => { + Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + vi.useFakeTimers(); + const frames = new Map(); + let frameId = 0; + vi.stubGlobal('requestAnimationFrame', (callback: FrameRequestCallback) => { + frames.set(++frameId, callback); + return frameId; + }); + vi.stubGlobal('cancelAnimationFrame', (id: number) => frames.delete(id)); + const events = new EventTarget(); + const companion = new WindowCompanion({ + context: async () => ({ profileId: 'qa' }), + source: () => ({ addEventListener: events.addEventListener.bind(events), close() {} }), + }); + const send = (type: string, value: unknown) => + events.dispatchEvent(new MessageEvent(type, { data: JSON.stringify(value) })); + const baseline = (paused = false) => + send('companion/baseline', { + profileId: 'qa', + bot: { slug: 'ada', name: 'Ada', paused }, + activity: { generation: 'host', revision: 0, bots: [] }, + }); + await companion.start(); + companion.select('ada'); + baseline(); + const sound = new CompanionSound(); + const play = vi.spyOn(sound, 'play'); + const stop = vi.spyOn(sound, 'stop'); + const node = document.createElement('div'); + document.body.append(node); + const root = createRoot(node); + const advanceFrames = async (count: number) => { + for (let index = 0; index < count; index++) { + await act(() => { + vi.advanceTimersByTime(16); + const callbacks = [...frames.values()]; + frames.clear(); + for (const callback of callbacks) callback(performance.now()); + }); + } + }; + const visibility = vi.spyOn(document, 'hidden', 'get'); + try { + await act(() => + root.render( + createElement(WindowCompanionView, { + companion, + sound, + openDm() {}, + openAttention() {}, + openChannel() {}, + t: zhTranslate, + }), + ), + ); + await act(() => + send('companion/message', { + generation: 'host', + botId: 'ada', + channelId: 'dm', + channelName: 'Ada', + messageId: 'new', + body: 'abcdefghijklmno', + source: 'own-dm', + }), + ); + expect(play).not.toHaveBeenCalled(); + await advanceFrames(4); + expect(play).toHaveBeenCalledOnce(); + const shown = companion.getSnapshot().cards[0]!.shown; + visibility.mockReturnValue(true); + await act(() => document.dispatchEvent(new Event('visibilitychange'))); + expect(stop).toHaveBeenLastCalledWith('ada'); + await advanceFrames(100); + expect(play).toHaveBeenCalledOnce(); + expect(companion.getSnapshot().cards[0]!.shown).toBe(shown); + visibility.mockReturnValue(false); + await act(() => document.dispatchEvent(new Event('visibilitychange'))); + await advanceFrames(4); + expect(play).toHaveBeenCalledTimes(2); + expect(companion.getSnapshot().cards[0]!.shown - shown).toBeLessThanOrEqual(2); + await act(() => events.dispatchEvent(new Event('error'))); + expect(stop).toHaveBeenLastCalledWith('ada'); + await advanceFrames(4); + expect(play).toHaveBeenCalledTimes(2); + await act(() => baseline(true)); + expect(stop).toHaveBeenLastCalledWith('ada'); + await advanceFrames(4); + expect(play).toHaveBeenCalledTimes(2); + stop.mockClear(); + await act(() => root.unmount()); + expect(stop).toHaveBeenLastCalledWith('ada'); + expect(frames.size).toBe(0); + } finally { + await act(() => root.unmount()); + companion.dispose(); + sound.dispose(); + node.remove(); + visibility.mockRestore(); + vi.unstubAllGlobals(); + vi.useRealTimers(); + } +}); + +it('keeps the Avatar tether aligned through resize and the first reduced-motion release commit', async () => { Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); const frames = new Map(); let frameId = 0; @@ -55,8 +253,8 @@ it('projects the owning Avatar screen point into an offset overlay and remeasure if (!this.classList.contains('bh-persona-avatar')) return stage; const surface = this.closest('.bh-companion')!; return new DOMRect( - stage.left + Number.parseFloat(surface.style.left), - stage.bottom - Number.parseFloat(surface.style.bottom) - 96, + stage.left + Number.parseFloat(companionPosition(surface).left), + stage.bottom - Number.parseFloat(companionPosition(surface).bottom) - 96, 96, 96, ); @@ -111,10 +309,35 @@ it('projects the owning Avatar screen point into an offset overlay and remeasure ); stage = new DOMRect(100, 60, 320, 500); await act(() => window.dispatchEvent(new Event('resize'))); - expect(node.querySelector('.bh-companion')?.style.left).toBe('216px'); + expect(companionPosition(node.querySelector('.bh-companion')!).left).toBe('216px'); expect(node.querySelector('.bh-companion-tether path')?.getAttribute('d')).toMatch( /^M 264 404 /u, ); + await act(() => { + document.documentElement.dataset['botharnessMotion'] = 'reduce'; + }); + const character = node.querySelector('.bh-companion-character')!; + character.setPointerCapture = vi.fn(); + character.releasePointerCapture = vi.fn(); + const pointer = (type: string, x: number, y: number) => { + const event = new MouseEvent(type, { bubbles: true, clientX: x, clientY: y, button: 0 }); + Object.defineProperty(event, 'pointerId', { value: 1 }); + character.dispatchEvent(event); + }; + await act(() => pointer('pointerdown', 264, 450)); + await act(() => pointer('pointermove', 164, 150)); + await act(frame); + expect(companionPosition(node.querySelector('.bh-companion')!).bottom).toBe( + '300px', + ); + expect(node.querySelector('.bh-companion-tether path')?.getAttribute('d')).toMatch( + /^M 164 104 /u, + ); + await act(() => pointer('pointerup', 164, 150)); + expect(companionPosition(node.querySelector('.bh-companion')!).bottom).toBe('0px'); + expect(node.querySelector('.bh-companion-tether path')?.getAttribute('d')).toMatch( + /^M 164 404 /u, + ); const visibility = vi.spyOn(document, 'hidden', 'get').mockReturnValue(true); await act(() => document.dispatchEvent(new Event('visibilitychange'))); expect(frames.size).toBe(0); @@ -125,6 +348,7 @@ it('projects the owning Avatar screen point into an offset overlay and remeasure node.remove(); measurement.mockRestore(); vi.unstubAllGlobals(); + delete document.documentElement.dataset['botharnessMotion']; } }); @@ -321,6 +545,114 @@ it('enables Group playback from the native menu and explains why a Bot-only sour } }); +it.each([ + [100, 0, false], + [200, 0, false], + [500, 0, false], + [16, 1000, false], + [16, 0, true], +])( + 'returns a fast upward throw with %i ms frames, %i ms before release and late movement %s', + async (interval, beforeRelease, lateMovement) => { + Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); + vi.useFakeTimers(); + const frames = new Map(); + let frameId = 0; + vi.stubGlobal('requestAnimationFrame', (callback: FrameRequestCallback) => { + frames.set(++frameId, callback); + return frameId; + }); + vi.stubGlobal('cancelAnimationFrame', (id: number) => frames.delete(id)); + const measurement = vi + .spyOn(HTMLElement.prototype, 'getBoundingClientRect') + .mockReturnValue(new DOMRect(0, 0, 1000, 800)); + const events = new EventTarget(); + const companion = new WindowCompanion({ + context: async () => ({ profileId: 'qa' }), + source: () => ({ addEventListener: events.addEventListener.bind(events), close() {} }), + }); + await companion.start(); + companion.select('ada'); + events.dispatchEvent( + new MessageEvent('companion/baseline', { + data: JSON.stringify({ + profileId: 'qa', + bot: { slug: 'ada', name: 'Ada', paused: false }, + activity: { generation: 'host', revision: 0, bots: [] }, + }), + }), + ); + const node = document.createElement('div'); + document.body.append(node); + const root = createRoot(node); + const openDm = vi.fn(); + try { + await act(() => + root.render( + createElement(WindowCompanionView, { + companion, + openDm, + openAttention() {}, + openChannel() {}, + t: zhTranslate, + }), + ), + ); + const character = node.querySelector('.bh-companion-character'); + const surface = node.querySelector('.bh-companion'); + if (!(character instanceof HTMLButtonElement) || !(surface instanceof HTMLElement)) + throw new Error('Missing companion controls'); + character.setPointerCapture = vi.fn(); + character.releasePointerCapture = vi.fn(); + const pointer = (type: string, y: number) => { + const event = new MouseEvent(type, { bubbles: true, clientX: 700, clientY: y, button: 0 }); + Object.defineProperty(event, 'pointerId', { value: 1 }); + character.dispatchEvent(event); + }; + await act(() => pointer('pointerdown', 750)); + await act(() => { + vi.advanceTimersByTime(beforeRelease); + vi.advanceTimersByTime(16); + pointer('pointermove', 550); + }); + await act(() => { + vi.advanceTimersByTime(16); + pointer('pointermove', 300); + pointer('pointerup', 300); + if (lateMovement) { + pointer('lostpointercapture', 300); + pointer('pointermove', 290); + character.click(); + } + }); + expect(surface.dataset['motion']).toBe('fall'); + const releasedBottom = Number.parseFloat(companionPosition(surface).bottom); + const heights: number[] = []; + for (let index = 0; index < 3000 / interval; index++) { + await act(() => { + vi.advanceTimersByTime(interval); + const due = [...frames.values()]; + frames.clear(); + due.forEach((callback) => callback(performance.now())); + }); + heights.push(Number.parseFloat(companionPosition(surface).bottom)); + } + expect(heights.some((height) => height > releasedBottom)).toBe(true); + expect(heights.slice(0, 2000 / interval).some((height) => height === 0)).toBe(true); + expect(surface.dataset['motion']).toBe('rest'); + expect(companionPosition(surface).bottom).toBe('0px'); + expect(openDm).not.toHaveBeenCalled(); + } finally { + await act(() => root.unmount()); + companion.dispose(); + node.remove(); + measurement.mockRestore(); + vi.useRealTimers(); + vi.unstubAllGlobals(); + } + }, +); + it('drags inside the shell, lands on the floor without opening DM, persists keyboard movement and releases frames', async () => { Object.assign(globalThis, { IS_REACT_ACT_ENVIRONMENT: true }); vi.useFakeTimers(); @@ -331,14 +663,18 @@ it('drags inside the shell, lands on the floor without opening DM, persists keyb return frameId; }); vi.stubGlobal('cancelAnimationFrame', (id: number) => frames.delete(id)); - let intersect: ((entries: { isIntersecting: boolean }[]) => void) | undefined; + let intersect: ((entries: { target: Element; isIntersecting: boolean }[]) => void) | undefined; const disconnected = vi.fn(); vi.stubGlobal( 'IntersectionObserver', class { - constructor(private readonly callback: (entries: { isIntersecting: boolean }[]) => void) {} - observe(target: Element) { - if (target.classList.contains('bh-companion')) intersect = this.callback; + constructor( + private readonly callback: ( + entries: { target: Element; isIntersecting: boolean }[], + ) => void, + ) {} + observe() { + intersect = this.callback; } disconnect = disconnected; }, @@ -375,11 +711,15 @@ it('drags inside the shell, lands on the floor without opening DM, persists keyb document.body.append(node); const root = createRoot(node); const openDm = vi.fn(); + const sound = new CompanionSound(); + const stop = vi.spyOn(sound, 'stop'); + const interact = vi.spyOn(sound, 'interact'); try { await act(() => root.render( createElement(WindowCompanionView, { companion: owner, + sound, openDm, openAttention() {}, openChannel() {}, @@ -389,8 +729,14 @@ it('drags inside the shell, lands on the floor without opening DM, persists keyb ); const character = node.querySelector('.bh-companion-character'); const surface = node.querySelector('.bh-companion'); - if (!(character instanceof HTMLButtonElement) || !(surface instanceof HTMLElement)) + const stage = node.querySelector('.bh-companion-stage'); + if ( + !(character instanceof HTMLButtonElement) || + !(surface instanceof HTMLElement) || + !(stage instanceof HTMLElement) + ) throw new Error('Missing companion controls'); + expect(node.querySelector('.bh-companion-activity')).toBeNull(); await act(() => character.focus()); expect(owner.getSnapshot().reading).toBe(true); await act(() => @@ -404,9 +750,9 @@ it('drags inside the shell, lands on the floor without opening DM, persists keyb ); expect(node.querySelector('[aria-expanded="true"]')).toBeNull(); expect(document.activeElement).toBe(character); - await act(() => intersect!([{ isIntersecting: false }])); + await act(() => intersect!([{ target: stage, isIntersecting: false }])); expect(frames.size).toBe(0); - await act(() => intersect!([{ isIntersecting: true }])); + await act(() => intersect!([{ target: stage, isIntersecting: true }])); expect(frames.size).toBe(1); const visibility = vi.spyOn(document, 'hidden', 'get'); visibility.mockReturnValue(true); @@ -426,43 +772,65 @@ it('drags inside the shell, lands on the floor without opening DM, persists keyb Object.defineProperty(event, 'pointerId', { value: id }); character.dispatchEvent(event); }; + const advanceFrames = async (count: number) => { + for (let index = 0; index < count; index++) { + await act(() => { + vi.advanceTimersByTime(16); + const callbacks = [...frames.values()]; + frames.clear(); + for (const callback of callbacks) callback(performance.now()); + }); + } + }; await act(() => pointer('pointerdown', 700, 750)); + await act(() => pointer('pointermove', 702, 750)); + expect(interact).not.toHaveBeenCalled(); await act(() => pointer('pointermove', 400, 400)); - expect(character.style.transform).toMatch(/^rotate\(-[\d.]+deg\)/u); + expect(interact).toHaveBeenLastCalledWith('ada', { kind: 'grab', strength: 1 }); + await advanceFrames(2); + expect(Number.parseFloat(character.style.rotate)).toBeLessThan(0); + const beforeReversal = character.style.rotate; await act(() => pointer('pointermove', 450, 400)); - expect(character.style.transform).toMatch(/^rotate\([\d.]+deg\)/u); + expect(character.style.rotate).toBe(beforeReversal); + for (let index = 0; index < 12; index++) { + await act(() => pointer('pointermove', 450 + index * 5, 400)); + await advanceFrames(1); + } + expect(Number.parseFloat(character.style.rotate)).toBeGreaterThan(0); await act(() => pointer('pointermove', 400, 400)); - const lifted = Number.parseFloat(surface.style.bottom); + const lifted = Number.parseFloat(companionPosition(surface).bottom); expect(lifted).toBeGreaterThan(12); - const draggedLeft = surface.style.left; + const draggedLeft = companionPosition(surface).left; await act(() => window.dispatchEvent(new Event('resize'))); - expect(Number.parseFloat(surface.style.bottom)).toBe(lifted); + expect(Number.parseFloat(companionPosition(surface).bottom)).toBe(lifted); await act(() => pointer('pointermove', 400, 400)); - expect(surface.style.left).toBe(draggedLeft); + expect(companionPosition(surface).left).toBe(draggedLeft); await act(() => { pointer('pointerdown', 400, 400, 2); pointer('pointermove', 800, 100, 2); pointer('pointercancel', 800, 100, 2); pointer('lostpointercapture', 800, 100, 2); }); - expect(surface.style.left).toBe(draggedLeft); + expect(companionPosition(surface).left).toBe(draggedLeft); expect(surface.dataset['motion']).toBe('drag'); await act(() => { pointer('pointerup', 400, 400); character.click(); }); - expect(Number.parseFloat(surface.style.bottom)).toBe(lifted); + expect(Number.parseFloat(companionPosition(surface).bottom)).toBe(lifted); await act(() => window.dispatchEvent(new Event('resize'))); - expect(Number.parseFloat(surface.style.bottom)).toBe(lifted); + expect(Number.parseFloat(companionPosition(surface).bottom)).toBe(lifted); expect(surface.dataset['motion']).toBe('fall'); await act(() => { document.documentElement.dataset['botharnessMotion'] = 'reduce'; }); - expect(surface.style.bottom).toBe('0px'); - expect(character.style.transform).toBe('rotate(0deg) scale(1, 1)'); + expect(companionPosition(surface).bottom).toBe('0px'); + expect(character.style.rotate).toBe('0deg'); + expect(character.style.transform).toBe('scale(1, 1)'); await act(() => { delete document.documentElement.dataset['botharnessMotion']; }); + interact.mockClear(); await act(() => { pointer('pointerdown', 400, 750); pointer('pointermove', 400, 750 - lifted); @@ -479,7 +847,7 @@ it('drags inside the shell, lands on the floor without opening DM, persists keyb frames.clear(); for (const callback of callbacks) callback(performance.now()); }); - positions.push(Number.parseFloat(surface.style.bottom)); + positions.push(Number.parseFloat(companionPosition(surface).bottom)); } expect(positions.some((bottom) => bottom > 0 && bottom < lifted)).toBe(true); expect(positions.every((bottom) => bottom >= 0 && bottom <= window.innerHeight - 120)).toBe( @@ -488,22 +856,60 @@ it('drags inside the shell, lands on the floor without opening DM, persists keyb const firstContact = positions.findIndex((bottom) => bottom === 0); expect(firstContact).toBeGreaterThan(0); expect(positions.slice(firstContact + 1).some((bottom) => bottom > 0)).toBe(true); - expect(surface.style.bottom).toBe('0px'); + expect(companionPosition(surface).bottom).toBe('0px'); expect(openDm).not.toHaveBeenCalled(); + expect(interact.mock.calls.filter(([, event]) => event.kind === 'land')).toHaveLength(1); + await act(() => owner.reading(false)); + await act(() => + events.dispatchEvent( + new MessageEvent('companion/message', { + data: JSON.stringify({ + generation: 'host', + botId: 'ada', + channelId: 'dm', + channelName: 'Ada', + messageId: 'offscreen-playback', + body: 'Keep this text paused while the thrown character is outside the visible area.', + source: 'own-dm', + }), + }), + ), + ); + await act(() => owner.reading(true)); + for (const release of ['pointerup', 'pointercancel', 'lostpointercapture']) { + await act(() => { + pointer('pointerdown', 400, 750); + pointer('pointermove', 450, -50); + pointer(release, 450, -50); + }); + await advanceFrames(1); + await act(() => intersect!([{ target: surface, isIntersecting: false }])); + expect(stop).toHaveBeenLastCalledWith('ada'); + interact.mockClear(); + const shown = owner.getSnapshot().cards[0]!.shown; + expect(Number.isFinite(Number.parseFloat(companionPosition(surface).bottom))).toBe(true); + await advanceFrames(240); + expect(surface.dataset['motion']).toBe('rest'); + expect(companionPosition(surface).bottom).toBe('0px'); + expect(owner.getSnapshot().cards[0]!.shown).toBe(shown); + expect(interact).not.toHaveBeenCalled(); + } + await act(() => intersect!([{ target: surface, isIntersecting: true }])); await act(() => character.dispatchEvent(new KeyboardEvent('keydown', { key: 'ArrowRight', bubbles: true })), ); expect(owner.getSnapshot().selection?.position).toBeCloseTo( - Number.parseFloat(surface.style.left) / 896, + Number.parseFloat(companionPosition(surface).left) / 896, ); document.documentElement.dataset['botharnessMotion'] = 'reduce'; await act(() => { pointer('pointerdown', 400, 750); pointer('pointermove', 700, 200); }); - expect(character.style.transform).toBe('rotate(0deg) scale(1, 1)'); + expect(character.style.rotate).toBe('0deg'); + expect(character.style.transform).toBe('scale(1, 1)'); await act(() => pointer('pointerup', 700, 200)); - expect(surface.style.bottom).toBe('0px'); + expect(companionPosition(surface).bottom).toBe('0px'); await act(() => vi.runOnlyPendingTimers()); delete document.documentElement.dataset['botharnessMotion']; await act(() => character.click()); @@ -514,6 +920,16 @@ it('drags inside the shell, lands on the floor without opening DM, persists keyb }); expect(surface.textContent).toContain('同步中断'); expect(frames.size).toBeGreaterThan(0); + const focusVisible = vi.spyOn(character, 'matches').mockReturnValue(false); + await act(() => + character.dispatchEvent( + new MouseEvent('pointerout', { bubbles: true, relatedTarget: document.body }), + ), + ); + await act(() => vi.advanceTimersByTime(800)); + expect(document.activeElement).toBe(character); + expect(owner.getSnapshot().reading).toBe(false); + focusVisible.mockRestore(); await act(() => root.unmount()); expect(frames.size).toBe(0); expect(disconnected).toHaveBeenCalled(); diff --git a/packages/client/test/window-companion.test.ts b/packages/client/test/window-companion.test.ts index d88243d1bb..fce4af3a0d 100644 --- a/packages/client/test/window-companion.test.ts +++ b/packages/client/test/window-companion.test.ts @@ -254,6 +254,46 @@ it('bounds parallel cards at twenty, freezes reading order, continues typing and expect(owner.getSnapshot().cards).toHaveLength(0); }); +it('shows fresh arrivals when focused reading has no cards and releases waiting arrivals after the last dismissal', async () => { + const events = new EventTarget(); + const owner = new WindowCompanion({ + context: async () => ({ profileId: 'qa' }), + source: () => ({ addEventListener: events.addEventListener.bind(events), close() {} }), + }); + controllers.push(owner); + await owner.start(); + owner.select('ada'); + const send = (type: string, value: unknown) => + events.dispatchEvent(new MessageEvent(type, { data: JSON.stringify(value) })); + send('companion/baseline', { + profileId: 'qa', + bot: { slug: 'ada', name: 'Ada', paused: false }, + activity: { generation: 'host', revision: 0, bots: [] }, + }); + const reply = (messageId: string) => + send('companion/message', { + generation: 'host', + botId: 'ada', + messageId, + channelId: 'dm', + channelName: 'Ada', + body: 'A new reply', + }); + reply('first'); + owner.reading(true); + owner.dismiss('first', 'dm'); + expect(owner.getSnapshot()).toMatchObject({ reading: true, cards: [], pending: 0 }); + reply('second'); + expect(owner.getSnapshot().cards.map((card) => card.messageId)).toEqual(['second']); + expect(owner.getSnapshot().pending).toBe(0); + reply('third'); + expect(owner.getSnapshot().cards.map((card) => card.messageId)).toEqual(['second']); + expect(owner.getSnapshot().pending).toBe(1); + owner.dismiss('second', 'dm'); + expect(owner.getSnapshot().cards.map((card) => card.messageId)).toEqual(['third']); + expect(owner.getSnapshot()).toMatchObject({ reading: true, pending: 0 }); +}); + it('reveals whole emoji and combining graphemes and clips previews on a complete boundary', async () => { const events = new EventTarget(); const owner = new WindowCompanion({ diff --git a/packages/client/test/window-companions-view.test.ts b/packages/client/test/window-companions-view.test.ts index f4c66ab307..771021cdb6 100644 --- a/packages/client/test/window-companions-view.test.ts +++ b/packages/client/test/window-companions-view.test.ts @@ -6,6 +6,8 @@ import { WindowCompanions } from '../src/client/window-companions.js'; import { WindowCompanionsView, CompanionSettings } from '../src/client/window-companions-view.js'; import { zhTranslate } from '../src/client/locale.js'; import { GroupChannelHeader } from '../src/client/group-channel-header.js'; +import { CompanionSound } from '../src/client/companion-sound.js'; +import { companionPosition } from './companion-position.js'; vi.mock('@deepseek-ai/dsh-client-ui-primitives', () => ({ Menu: ({ @@ -36,6 +38,25 @@ vi.mock('@deepseek-ai/dsh-client-ui-primitives', () => ({ : null, ), Input: (props: Record) => createElement('input', props), + Switch: ({ + checked, + onChange, + label, + disabled, + }: { + checked: boolean; + onChange(value: boolean): void; + label: string; + disabled: boolean; + }) => + createElement('button', { + type: 'button', + role: 'switch', + 'aria-label': label, + 'aria-checked': checked, + disabled, + onClick: () => onChange(!checked), + }), Button: ({ children, size: _size, @@ -69,10 +90,10 @@ it('pins independently, exposes right-click controls and applies global bounded .spyOn(HTMLElement.prototype, 'getBoundingClientRect') .mockImplementation(function (this: HTMLElement) { if (this.classList.contains('bh-persona-avatar')) { - const companion = this.closest('.bh-companion'); + const position = companionPosition(this.closest('.bh-companion')!); return new DOMRect( - Number.parseFloat(companion?.style.left ?? '0'), - window.innerHeight - Number.parseFloat(companion?.style.bottom ?? '0') - 96, + Number.parseFloat(position.left), + window.innerHeight - Number.parseFloat(position.bottom) - 96, 96, 96, ); @@ -89,6 +110,9 @@ it('pins independently, exposes right-click controls and applies global bounded const node = document.createElement('div'); document.body.append(node); const root = createRoot(node); + const listeners = vi.spyOn(document, 'addEventListener'); + const removals = vi.spyOn(document, 'removeEventListener'); + const unlock = vi.spyOn(CompanionSound.prototype, 'unlock'); try { await act(() => root.render( @@ -127,6 +151,9 @@ it('pins independently, exposes right-click controls and applies global bounded ), ); const pins = node.querySelectorAll('.bh-companion-pin'); + expect(listeners).toHaveBeenCalledWith('pointerdown', expect.any(Function)); + document.dispatchEvent(new MouseEvent('pointerdown', { bubbles: true })); + expect(unlock).not.toHaveBeenCalled(); expect(pins).toHaveLength(2); expect(pins[0]!.getAttribute('aria-label')).toContain('Ada'); expect(pins[1]!.getAttribute('aria-label')).toContain('Grace'); @@ -159,7 +186,7 @@ it('pins independently, exposes right-click controls and applies global bounded generation: 'host', revision: 1, bots: [ - { slug: 'ada', state: 'idle' }, + { slug: 'ada', state: 'working' }, { slug: 'grace', state: 'thinking' }, ], }, @@ -172,10 +199,11 @@ it('pins independently, exposes right-click controls and applies global bounded for (const callback of [...frames.values()]) callback(performance.now()); }); const bubbles = [...node.querySelectorAll('.bh-companion-activity')]; + expect(bubbles).toHaveLength(2); const positions = bubbles.map((bubble) => ({ x: Number.parseFloat(bubble.style.left) + - Number.parseFloat(bubble.closest('.bh-companion')!.style.left), + Number.parseFloat(companionPosition(bubble.closest('.bh-companion')!).left), y: Number.parseFloat(bubble.style.bottom), })); expect( @@ -230,8 +258,9 @@ it('pins independently, exposes right-click controls and applies global bounded const bounds = (botId: string) => { const character = node.querySelector(`[data-bot="${botId}"]`)!; const cards = character.querySelector('.bh-companion-cards')!; - const left = Number.parseFloat(character.style.left) + Number.parseFloat(cards.style.left); - const bottom = Number.parseFloat(cards.style.bottom); + const position = companionPosition(character); + const left = Number.parseFloat(position.left) + Number.parseFloat(cards.style.left); + const bottom = Number.parseFloat(position.bottom) + Number.parseFloat(cards.style.bottom); return { left, right: left + 320, @@ -321,11 +350,22 @@ it('pins independently, exposes right-click controls and applies global bounded ); expect(owner.getSnapshot().capacity).toEqual({ layers: 2, retention: 2 }); expect(owner.get('grace')!.getSnapshot().capacity.retention).toBe(2); + const speech = node.querySelector('[role="switch"]')!; + expect(speech.getAttribute('aria-checked')).toBe('false'); + await act(() => speech.click()); + expect(owner.getSnapshot().speechSound).toBe(true); + expect(speech.getAttribute('aria-checked')).toBe('true'); + await act(() => speech.click()); + expect(owner.getSnapshot().speechSound).toBe(false); await act(() => pins[0]!.click()); expect(node.querySelector('[data-bot="ada"]')).toBeNull(); expect(node.querySelector('[data-bot="grace"]')).not.toBeNull(); } finally { await act(() => root.unmount()); + expect(removals).toHaveBeenCalledWith('pointerdown', expect.any(Function)); + listeners.mockRestore(); + removals.mockRestore(); + unlock.mockRestore(); node.remove(); owner.dispose(); measurement.mockRestore(); diff --git a/packages/client/test/window-companions.test.ts b/packages/client/test/window-companions.test.ts index 66177d9df7..a181836eca 100644 --- a/packages/client/test/window-companions.test.ts +++ b/packages/client/test/window-companions.test.ts @@ -1,6 +1,48 @@ import { expect, it, vi } from 'vitest'; import { WindowCompanions } from '../src/client/window-companions.js'; +it('defaults to silent and restores only an explicit Profile-scoped speech-sound choice without changing subscriptions', async () => { + const stored = new Map(); + const storage = { + getItem: (key: string) => stored.get(key) ?? null, + setItem: (key: string, value: string) => { + stored.set(key, value); + }, + }; + const source = vi.fn(() => ({ addEventListener() {}, close() {} })); + const create = (profileId: string) => + new WindowCompanions({ + storage, + context: async () => ({ profileId }), + source, + update: async () => {}, + }); + const first = create('one'); + await first.start(); + expect(first.getSnapshot().speechSound).toBe(false); + first.configureSpeechSound(true); + expect(first.getSnapshot().speechSound).toBe(true); + first.configureCapacity({ layers: 2, retention: 10 }); + first.dispose(); + const restored = create('one'); + const other = create('two'); + try { + await restored.start(); + await other.start(); + expect(restored.getSnapshot()).toMatchObject({ + speechSound: true, + capacity: { layers: 2, retention: 10 }, + }); + expect(other.getSnapshot().speechSound).toBe(false); + restored.configureSpeechSound(false); + expect(JSON.parse(stored.get('botharness/companions/v2/one')!).speechSound).toBe(false); + expect(source).not.toHaveBeenCalled(); + } finally { + restored.dispose(); + other.dispose(); + } +}); + it('does not let an old Activity acknowledgement cancel a failed removal retry', async () => { vi.useFakeTimers(); const events = new EventTarget(); diff --git a/packages/core/test/companion-recovery.test.ts b/packages/core/test/companion-recovery.test.ts index 01140f02fc..a1bca674e4 100644 --- a/packages/core/test/companion-recovery.test.ts +++ b/packages/core/test/companion-recovery.test.ts @@ -225,9 +225,13 @@ it('keeps reading progress and deduplicates recovery through the real feed into await receive(); await vi.waitFor(() => expect(updates).toBe(3)); await receive(); + expect(child.getSnapshot()).toMatchObject({ reading: true, cards: [], pending: 0 }); await commit('future-after-enable'); await receive(); - expect(child.getSnapshot().pending).toBe(1); + expect(child.getSnapshot().pending).toBe(0); + expect(child.getSnapshot().cards.map((card) => card.messageId)).toEqual([ + 'future-after-enable', + ]); child.reading(false); expect(child.getSnapshot().cards.map((card) => card.messageId)).toEqual([ 'future-after-enable',