Skip to content

fix(tui): widen local-endpoint SSE open timeout to 180s - #130

Open
sky11023 wants to merge 2 commits into
Pinvou:pinvou3-cleanfrom
sky11023:fix/local-stream-open-timeout-20261029
Open

sky11023 wants to merge 2 commits into
Pinvou:pinvou3-cleanfrom
sky11023:fix/local-stream-open-timeout-20261029

Conversation

@sky11023

Copy link
Copy Markdown

问题

品悟 Agent(约 25K token 系统提示词)冷 prefill 时,本地模型在首字节前静默约 30 秒(TensorFold 双卡 854 tok/s 实测,2026-10-29 现场)。TUI 三个流式适配器(chat / responses / anthropic)共用 45 秒的流打开(响应头)超时——这个预算是为「云主机死掉要快速失败」设计的;对本地端点,长提示词冷启动合法地超过 45s,用户侧表现为「回答被中断,当前请求超时请重试」。dspark 等卡同场景可复现,非单卡特有。

修复

crates/tui/src/client/stream_entry.rs:

  • 新增 LOCAL_STREAM_OPEN_TIMEOUT = 180s 与 stream_open_timeout_for(base_url):endpoint_is_local()(loopback / localhost / RFC1918 / link-local / ULA)命中的 base URL 走 180s 打开预算,其余维持 45s。
  • chat / responses / anthropic 三个调用点全部切换到 stream_open_timeout_for。
  • 显式环境变量覆盖对两类端点仍然最高优先。
  • 块间空闲超时维持 900s 不变(已高于本地 120s 契约),改动只放宽「首个响应头」等待。

验证

  • 本地 cargo test -p codewhale-tui --lib(精确选择)3/3 通过:local_endpoints_get_the_wider_open_budget、env_override_wins_over_the_local_budget、stream_open_timeout_defaults_and_clamps_env_values。
  • 该提交(cb577ba55)已被 pinvou3 双机从机集成分支以 gitlink 引用并完成过整包构建装机验证(integrated.20261029.2,.10 设备 worker CLI 实测 exit 0)。请按原样合并,勿 rebase 改写哈希。

DCO:Signed-off-by 已带。

@asto18089 asto18089 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First formal review — #130 fix(tui): widen local-endpoint SSE open timeout to 180s

Thanks for the PR and for the unusually well-documented reasoning (field data, per-class budget, and an env escape hatch preserved for both classes). Reviewed file by file against pinvou3-clean (fe90f72). No blockers found — this is a COMMENT review with minor items.

Per-file findings

crates/tui/Cargo.toml + Cargo.lock — url = "2"

  • OK / necessary. url 2.5.8 is already in Cargo.lock transitively (via reqwest), so this adds no new build nodes. Correct tool for the job: hand-parsing IPv6 bracket hosts is error-prone. Spec "2" resolves to the locked 2.5.8 and matches the neighboring direct-spec style (urlencoding = "2.1", regex = "1.11"). There is no workspace.dependencies entry for url, so a direct spec is consistent with convention.

crates/tui/src/client/stream_entry.rs — stream_open_timeout_for

  • Env override semantics are consistent with the existing path: presence check → delegate to stream_open_timeout() → same parse + clamp (5..=300, garbage falls back to 45s). The escape hatch still exceeds the local budget (up to 300s), so it works in both directions. Correct.
  • nit: with env set to a invalid value (e.g. abc), a local endpoint silently drops from 180s to 45s (presence-check wins, parse falls back to default). Edge case, conservative direction — fine to leave.

crates/tui/src/client/stream_entry.rs — endpoint_is_local

  • IPv4: loopback / RFC1918 / link-local via std predicates — correct.
  • IPv6: ::1 + ULA fc00::/7 — correct as far as it goes.
  • DNS names: only localhost / *.localhost count; a hostname that resolves to a private IP is deliberately not local (documented, no DNS on this path). This is the right conservative default (a cloud gateway behind a resolvable name would be misclassified the other way), but note it means LAN engines reachable only by hostname get 45s — only raw private IPs benefit.
  • Conservative-direction gaps, each falls back to today's 45s (no regression), listed so the author can decide whether to widen: (1) IPv4-mapped IPv6 ([::ffff:127.0.0.1]) is Ipv6 and not loopback → remote; (2) IPv6 link-local fe80::/10 → remote (v4 link-local is covered, v6 is not); (3) CGNAT 100.64.0.0/10 — i.e. Tailscale-style endpoints, a very common way to reach a home GPU box, → remote; (4) 0.0.0.0 unspecified → remote. Given validate_base_url_security explicitly invites "llama.cpp on a private IP" over HTTP with the insecure-http env, item (3) is the one I'd most consider following up on (non-blocking).

crates/tui/src/client/{anthropic,chat,responses}.rs — wiring

  • Complete. Verified by grep on base: exactly three stream_open_timeout() call sites exist (chat.rs:1346, anthropic.rs:354, responses.rs:269) — all three rewired with the identical comment; the only other match is doc text in client.rs:3028. The H1-fallback retry reuses the same StreamOpenRequest, so the widened budget carries into the retry. Per-chunk idle (900s) unchanged and already ≥ the local contract, as the commit message states.
  • nit: classification reads self.base_url while the anthropic open goes through messages_transport_url; in production the transport URL derives from base_url (the override fields are test-only), so this is fine today.

Tests

  • Not flaky: no sleeps; 180s is a constant comparison, not a timing assertion.
  • ENV_LOCK correctly serializes the two env-touching tests, and no pre-existing test in this module touches the env, so no cross-test race.
  • MINOR (hermeticity): local_endpoints_get_the_wider_open_budget calls stream_open_timeout_for, which reads ambient env. On a machine where CODEWHALE_STREAM_OPEN_TIMEOUT_SECS is exported, the == LOCAL_STREAM_OPEN_TIMEOUT assertions fail. Please clear the var under the lock at test start (or assert on endpoint_is_local only for the budget pair).
  • nit: in env_override_wins_over_the_local_budget, if anything panics between set_var and remove_var, the var leaks into the process for other tests. A drop-guard (or catch_unwind) would make cleanup panic-safe.

Process notes

  • DCO: Signed-off-by matches author/committer (l28756 <li.jiawen@h3c.com>) — DCO satisfied. The git name l28756 vs GitHub account sky11023 mismatch is noted; not a blocker, but aligning them avoids future verification confusion.
  • Behind by 2 (fe90f72f #114, f1eb10ef #125): zero file overlap (engine/skills/docs vs client/*, no Cargo manifests), GitHub reports MERGEABLE. Per CONTRIBUTING, no rebase requested — the Merge Queue will gate against latest main.
  • CI: no workflow runs exist for head cb577ba5 (nothing in the runs list, no check runs, combined status pending with zero statuses — the first-contribution approval gate). Nothing to approve via the API from this side; maintainers, please approve/trigger the workflow run so checks actually execute.

Conclusion

No BLOCKER, no MAJOR. MINOR items: test hermeticity vs ambient env; the enumerated conservative-direction classification gaps (CGNAT/Tailscale the most notable); optional panic-safe env cleanup. Comfortable to merge after a maintainer eyes the MINORs; comments-only, changes not requested.

@sky11023

sky11023 commented Nov 2, 2026

Copy link
Copy Markdown
Author

感谢首轮正式评审的细致核对(逐文件、突变与 CI 归因都做了)。三个 MINOR 已全部处置(ca079796c):

  1. 测试密封性:local_endpoints_get_the_wider_open_budget 现在在 ENV_LOCK 下先 remove_var 两个超时变量再断言——宿主机 export 过该变量也不再误红。
  2. panic 安全清理:env_override_wins_over_the_local_budget 改用 Drop 守卫(EnvReset),断言 panic 也会清掉变量,不再泄进进程。
  3. 分类缺口:endpoint_is_local 补齐三档——CGNAT 100.64.0.0/10(手写第二字节 64..=127 判定,is_shared() 在当前工具链还是 unstable feature ip)、IPv6 链路本地 fe80::/10、IPv4-mapped v6(按 v4 载荷重判)。0.0.0.0 作为目的地刻意维持远端(见注释:不该有人拨它,放宽只会掩盖调用方 bug)——若您倾向也纳入可以说一声,一行的事。

验证:三测试精确名单 3/3 绿;cargo fmt --check 绿。(跑宽过滤时 config::tests 那条已知编排型测试偶发红,与本次改动无关——不碰任何 config 变量。)

DCO 备注:git 身份 l28756 与 GitHub 账号 sky11023 的不一致已知,后续提交会对齐。

@github-actions

github-actions Bot commented Nov 4, 2026

Copy link
Copy Markdown

Thanks @sky11023 for taking the time to contribute.

This repository is observing a maintainer-managed PR intake gate in dry-run mode, so this pull request is staying open. This note helps maintainers prepare the allowlist before any enforcement is considered.

Please read CONTRIBUTING.md for the expected contribution shape. A maintainer can grant recurring PR access by commenting /lgtm on a pull request.

@asto18089
asto18089 force-pushed the fix/local-stream-open-timeout-20261029 branch from ca07979 to fa79c99 Compare November 4, 2026 08:11
l28756 added 2 commits November 5, 2026 10:53
Cloud-budget 45s header wait exists to fail fast on dead cloud hosts;
a local model (loopback/private) legitimately spends minutes before
response headers when it prefills a very long prompt cold — pinvou3
field data (TensorFold dual-GPU, 2026-10-29): ~25K-token agent prompt
sits silent ~30s at 854 tok/s first-pass. endpoint_is_local() routes
loopback/localhost/RFC1918/link-local/ULA base URLs to a 180s budget
across all three streaming adapters; explicit env override still wins
for both classes. Per-chunk idle stays 900s (already >= the 120s
local contract).

Signed-off-by: l28756 <li.jiawen@h3c.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants