Skip to content

feat(tui): default agent-turn temperature to 0.3 - #132

Open
sky11023 wants to merge 2 commits into
Pinvou:pinvou3-cleanfrom
sky11023:fix/agent-default-temperature-20261102
Open

sky11023 wants to merge 2 commits into
Pinvou:pinvou3-cleanfrom
sky11023:fix/agent-default-temperature-20261102

Conversation

@sky11023

@sky11023 sky11023 commented Nov 2, 2026 •

Copy link
Copy Markdown

问题

Agent 回合的 MessageRequest 从不显式下发采样参数,多数推理服务端按 temperature=1.0 处理——随机性最大,且实测会略降 MTP 类引擎的投机接受率(TensorFold 10-30 性能报告②:默认 1.0 vs 低温的接受率差异)。

修复(2026-11-02 拍板)

acp_server.rs 的 Agent 回合请求构造处:temperature: None → Some(DEFAULT_AGENT_TEMPERATURE),拍板值 0.3(兼顾确定性与多样性),以具名常量+钉测试防漂移。

  • 只影响 Agent 回合;缓存预热、翻译、压缩等内部请求维持各自既有值(预热显式 off reasoning,翻译非交互)。
  • 提供方固定采样契约不受影响:apply_kimi_code_fixed_sampling 等按 provider 剥离 temperature 的既有逻辑在序列化之后仍生效。
  • 副作用说明:本地引擎卡片的 LMM_TEMPERATURE 旋钮改的是服务端缺省,显式请求值会覆盖它——需要引擎侧自定义采样时应在模型配置层显式设置(未来引入用户级采样配置后由用户值优先)。

验证

  • cargo check -p codewhale-tui 绿;cargo fmt --check 绿;clippy 无新增(基线存在与本改动无关的历史 too_many_arguments 项)。
  • 焦点测试 agent_turn_default_temperature_is_the_ratified_value 通过;序列化契约(Some(t) → body.temperature)由 client 侧既有 Kimi 固定采样用例覆盖。
  • 基点 cb577ba55(= PR fix(tui): widen local-endpoint SSE open timeout to 180s #130 的 180s 超时修复,独立合并互不依赖)。

DCO:Signed-off-by 已带。

评审追补(r1 处置说明,见评论区)

  • 拍板溯源:LMM_TEMPERATURE 是品悟模型商店卡片脚本侧的引擎旋钮(随部署注入推理服务环境,定义见品悟仓 docs/模型卡片脚本规范.md §环境变量;本仓库不携带该规范,属跨仓引用)。2026-11-02 拍板记录在品悟侧任务书(双机性能优化②:品悟默认下发 temperature=0.x)。
  • 过渡期无用户级覆盖:已知并接受——品悟侧的用户级采样配置在同一批次规划中,落地前显式值不可配;引擎侧自定义采样的既有路径(模型配置层显式设置)仍可用,仅"缺省跟随引擎"的用法在 agent 回合被本 PR 覆盖。维持者已知悉。
  • 合并顺序:fix(tui): widen local-endpoint SSE open timeout to 180s #130 先合,本 PR rebase 后 diff 将只剩温度改动(评审已注明)。

@asto18089 asto18089 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First formal review — #132 feat(tui): default agent-turn temperature to 0.3 (stacked on #130)

Review of commit 9725d2cc (the second commit; the first is #130's cb577ba5, reviewed separately). One change requested — see MAJOR. Stacked note: this PR contains #130 in full. Merge order must be #130 → #132; any change to #130 (including its MINOR items) requires this branch to rebase, and after #130 merges a rebase would let this PR's diff show only the temperature change.

Per-file findings

crates/tui/src/acp_server.rs — temperature: None → Some(DEFAULT_AGENT_TEMPERATURE)

  • Scope verified: acp_server.rs:1995 is the only production construction site of the agent-turn MessageRequest (other sites are translation, cache-warmup copy, reviewer, and tests). The reviewer/guardian path already pins Some(0.0), so explicit sampling on agentic paths has precedent; translation/compaction keep server defaults. Scope is consistent.
  • Rejecting routes are protected, so Some(0.3) cannot leak where sampling is forbidden: the Anthropic adapter guards with anthropic_model_rejects_sampling (opus-4-7/4-8, *-5 families, fable, mythos); the Chat adapter strips temperature/top_p for exact direct-Moonshot-K3 and Kimi-Code membership routes; Responses writes temperature only when Some. Verified on base.
  • Justification: "servers treat absent temperature as 1.0" is accurate for OpenAI-compatible stacks (and their typical self-hosted reimplementations). The MTP/speculative-acceptance claim is attributed to an internal report I cannot verify from this repo — plausible, and clearly attributed, so acceptable.
  • MINOR (unverifiable cross-reference): LMM_TEMPERATURE appears nowhere in CodeWhale or pinvou3-app (grepped both). The comment's claim that the local-engine card knob sets the server-side default and that an explicit request value overrides it is standard API semantics, but the knob itself cannot be verified from this codebase. Please link the engine-side definition or the ratification record (issue/report) in the PR description so the "2026-11-02 拍板" is traceable.
  • MINOR (no escape hatch): once this lands, agent-turn sampling is hard-coded with no in-repo override until the promised user-level sampling config exists. Users who tuned sampling via engine-side defaults silently lose it on agent turns. The comment discloses this — good — but please confirm maintainers explicitly accept the interim state.
  • nit: the commit body is empty apart from the sign-off; the rationale lives only in code comments. A two-line body would help git log archaeology.

crates/tui/src/acp_server.rs — agent_turn_default_temperature_is_the_ratified_value

  • MAJOR: assert_eq!(DEFAULT_AGENT_TEMPERATURE, 0.3) is a pure tautology — it asserts a constant equals a literal and provides zero regression protection (it can only fail if someone edits the constant, which any diff would show). Please replace it with a behavioral assertion, e.g. build the ACP agent-turn request (or capture the wire body via the existing wiremock harness in client.rs tests) and assert body["temperature"] == 0.3 for a vanilla route, plus the negative space: no temperature on a sampling-rejecting route (anthropic_model_rejects_sampling hit) and on the Kimi-Code stripping route. That would also fix the comment's claim below.
  • MINOR: the test comment says the Some(t) → body.temperature serialization contract is covered by "client 侧 Kimi 固定采样用例" — those tests assert the opposite direction (Some(0.3) is stripped on that route). Pass-through is covered elsewhere (e.g. the wire assertion of temperature == 0.25 on the neighbor route); cite that instead.
  • nit: the test fn is inserted before use super::*; in mod tests. Compiles fine, unconventional placement — move it below the imports.

CI

No workflow runs exist for head 9725d2cc (runs list, check runs, and combined status all empty — first-contribution approval gate; zero runs in status=waiting). Nothing for me to approve via the API at my permission level — maintainers, please approve/trigger the workflow run for both this PR and #130.

Conclusion

  • MAJOR: tautological pinned test — replace with a wire-level assertion of the new default plus the rejecting-route negative cases.
  • MINOR: LMM_TEMPERATURE citation unverifiable in-repo (link it); no interim user override for the hard-coded 0.3 (confirm acceptance); test-comment cites the wrong covering test; empty commit body.
  • The behavior change itself is well-contained and correctly guarded on all wire paths. Requesting changes only for the test; once replaced and #130 has settled, this is ready. Merge order: #130 first, then this (rebased).

@asto18089 asto18089 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second review — rationale re-verification of the 0.3 default (2026-11-04)

The first review provisionally accepted the justification ("servers treat absent temperature as 1.0 … acceptable"). That line is superseded. The rationale was re-verified from primary sources — vendor API docs, official model cards and generation_config.json files, serving-stack source code, and the coding-agent ecosystem — via seven investigation lanes (four external, three in-repo). The premise of the change does not hold for either endpoint class this PR serves. The standing MAJOR (tautological test) from round 1 remains open.

What omitting temperature actually does (all verified 2026-11-03/04)

Endpoint Omitted → Vendor recommendation Explicit value
OpenAI (GPT-5.x / o-series) 1.0 1.0 — the only supported value ("Only the default (1) value is supported") non-1.0 → 400 on reasoning models
Anthropic Messages (2026-10-21) 1.0 1.0 — temperature/top_p/top_k removed: sending any of them "returns a 400 error, even at the parameter's default value"; omitted requests "sample with their defaults, including a temperature of 1.0" deprecated → 400
Google Gemini 3 1.0 1.0 — "we strongly recommend keeping the temperature parameter at its default value of 1.0"; below 1.0 "may lead to unexpected behavior, such as looping or degraded performance"; migration guide says to remove explicit low values warned against
DeepSeek API 1.0 V3.2 (final) ships generation_config.json temperature: 1.0, top_p: 0.95; SGLang's official V3.2 recipe: "temperature = 1.0, top_p = 0.95 (as recommended by DeepSeek)"; temperature "has no effect in thinking mode". (Legacy task table recommends 0.0 for coding/math — pre-thinking-era guidance for old non-thinking models) accepted
Qwen3 (DashScope) 0.7 non-thinking / 0.6 thinking = card recommendations (thinking 0.6/0.95, non-thinking 0.7/0.8); "DO NOT use greedy decoding" accepted
Zhipu GLM-4.6 1.0 1.0 ("we recommend using a sampling temperature of 1.0", + top_p 0.95 / top_k 40) accepted
Moonshot Kimi (k2.6+/K3/k2.7-code) n/a — parameter absent from the current OpenAPI spec fixed server-side (1.0 thinking/code, 0.6 k2.6 non-thinking); "Do not pass temperature explicitly when calling these models" error on any explicit value
MiniMax M2/M2.1 recommended 1.0 1.0 / top_p 0.95 / top_k 40; M3 ignores explicit values entirely ignored
vLLM ≥ 0.8.0 model's generation_config.json (--generation-config auto is the default since vllm#12622); 1.0 is only the last-resort fallback the model author's shipped values client value overrides generation config
SGLang ≥ 0.5.4 model generation config (--sampling-defaults model, the default); 1.0 fallback same client value wins
Ollama (Sep 2026+) / llama.cpp (Nov 2025+) GGUF-embedded / generation_config defaults; 0.8 only when the model ships none same client value wins

Sources: Anthropic migration 2026-10-21, Gemini 3 guide, DeepSeek parameter settings, DashScope defaults, GLM-4.6 card, Kimi models overview, MiniMax-M2.1 card, vLLM OpenAI-compat docs, SGLang sampling params, Ollama PR #16471, llama.cpp PR #17120.

Why the 0.3 rationale fails on both endpoint classes

  1. Cloud: "omitted → 1.0" is true, but 1.0 is the vendors' recommendation for current flagships — not a pathology to be corrected client-side. Gemini explicitly warns that sub-1.0 values degrade reasoning tasks; Anthropic removed the knob entirely; OpenAI reasoning models reject non-1.0. For older Anthropic models on unguarded routes, the PR now sends 0.3 where the server previously applied the per-model default — exactly what the Gemini migration guide tells integrators to stop doing.
  2. Local: "most inference servers treat missing temperature as 1.0" is not the current behavior of the stacks this PR's own first commit targets. vLLM ≥ 0.8.0, SGLang ≥ 0.5.4, and current Ollama/llama.cpp all prefer the model's shipped sampling defaults (that is what generation_config.json is for); 1.0 is only a last-resort fallback. An explicit client value overrides those vendor recommendations on every stack — so Some(0.3) forces, e.g., a correctly-served Qwen3-thinking from its recommended 0.6 down to 0.3, and DeepSeek-V3.2-Exp from 0.6 down to 0.3. The first commit of this very PR treats local endpoints as first-class; the second commit overrides their tuned defaults.
  3. No vendor recommends 0.3. The recommendation landscape is: 1.0 for the current flagship generation (OpenAI, Anthropic, Gemini, DeepSeek V3.2, GLM-4.6, Kimi thinking/code, MiniMax), or 0.6–0.7 for the reasoning-era minority (Qwen3, DeepSeek-Exp/V3.1/R1, Kimi K2-Instruct). 0.3 matches no published recommendation on any route it reaches.

What survives scrutiny

  • The MTP direction has one public corroboration: llama.cpp documents that with rejection-sampling verification "acceptance falls away as temperature rises" (PR #27694). But vLLM/SGLang MTP drafters are greedy by default (temperature-independent), and no vendor or stack publishes "keep temperature at X for accept rate" guidance — DeepSeek's own V3.2 recipe is 1.0/0.95 despite MTP. The cited TensorFold 10-30 report remains unverifiable from outside.
  • The wire guards hold: verified that Some(0.3) is stripped on Kimi-Code/direct-K3 membership routes (client/chat.rs:923-968) and anthropic_model_rejects_sampling routes (client/anthropic.rs:734-748); it reaches the wire on plain Chat Completions and older-Anthropic routes, and on Responses routes only for DeepSeek (client/responses.rs:140-147).

In-repo evidence against a hard-coded client default

  • This codebase already lived the failure mode: compaction once hard-coded 0.3, it leaked onto the Kimi Code membership route and failed the whole compaction pass; the fix removed the fixed value (compaction now sends None, pinned by tests at compaction.rs:2329/2660). PR #132 reintroduces the same pattern one path over.
  • Scope inconsistency: commit 9725d2cc0 touches only the ACP server path. The interactive TUI's own main turn (crates/core/src/request.rs:76), subagents, RLM, one-shot exec, and MCP chat all still send None — so the ratified "agent turns need stable output" premise is not applied to the product's own agent loop, only to external ACP clients.
  • LMM_TEMPERATURE is unverifiable: zero hits in both CodeWhale and pinvou3-app (grepped this round). The comment's override interaction is standard semantics (client value always wins), but the knob itself is not in any repo this PR touches — please link the engine-side definition or the ratification record. And since no user/model-level sampling config exists in-repo (no config field, no env var), any engine-side default is silently overridden with no escape hatch.
  • App-side prior art mandates omission: the pinvou3 settings-layer image probe deliberately omits temperature — "kimi-for-coding 等模型只接受默认值,显式传会 400" (settings.rs:811, test-pinned) — consistent with Kimi's current "do not pass explicitly" contract.
  • Ecosystem direction supports omission over hardcoding: Codex CLI omits temperature, OpenAI Agents SDK defaults to None, LangChain moved 0.7 → omitted, Claude Code has no sampling knob, and Gemini CLI is the notable (aging) exception that still sends explicit 0 on its main loop.

Requested changes

  1. Primary: revert temperature: Some(DEFAULT_AGENT_TEMPERATURE) back to None. Omitting is the only choice that yields the vendor-recommended value on every verified endpoint — cloud and local — and it matches where the API ecosystem is heading (parameter removal/rejection). If the "2026-11-02 拍板" decision survives this evidence, it needs re-ratification by maintainers with these sources on record.
  2. Alternative: if a deterministic agent-turn default is genuinely wanted, implement it as an explicit opt-in (model-level config field, the already-promised user-level sampling config), not a silent hardcode — and do not pick 0.3, which sits off every vendor's recommendation.
  3. Unchanged from round 1: replace the tautological constant test with wire-level assertions (0.3 present on a vanilla route; absent on the Anthropic-rejecting and Kimi stripping routes), fix the test comment's wrong cross-reference, and move the test below use super::*.
  4. Stacking: #130 has advanced to ca079796c while this PR still carries the old cb577ba5 — after #130 merges, rebase so the diff shows only the temperature change. CI remains empty on 9725d2cc0 (first-contribution gate) — maintainers, please still approve/trigger the workflow runs for #130 and #132.

@asto18089
asto18089 force-pushed the fix/agent-default-temperature-20261102 branch from 9725d2c to ad1b8ad Compare November 4, 2026 08:11
@sky11023

sky11023 commented Nov 4, 2026

Copy link
Copy Markdown
Author

r1 处置(head 97dd00aef)

MAJOR(同义反复钉测试):已替换为行为级三路由断言 agent_turn_temperature_survives_vanilla_and_is_stripped_on_sampling_fixed_routes——构造抽成 agent_turn_message_request(生产唯一 agent-turn 构造点),普通路由经生产 build_chat_wire_body 断 wire body temperature==0.3;Kimi-Code 会员路由经生产 apply_kimi_code_fixed_sampling 断剥除;Anthropic 拒绝采样族经新抽的 apply_anthropic_sampling 断两个采样参数都不带(含非拒绝族对照臂)。通道全部走生产函数,零重抄。

这个测试当场抓到一个真 bug:f32 采样参数直扩 f64 后,wire 上是 0.30000001192092896(17 位尾数)——旧的同义反复测试永远抓不到。已在序列化接缝修复:新增 sampling_wire_value(经 f32 最短十进制表示中转),chat 与 anthropic 两处落 wire 统一走它;既有 anthropic 测试钉的期望值恰是伪影值(Some(f64::from(0.7f32))=0.699999988079071),已更新为干净 0.7 并注明原委。

MINOR:①LMM_TEMPERATURE 溯源与②过渡期无覆盖的接受声明已写入 PR 正文(见「评审追补」节);③测试注释的错误引用随测试重写消除;④提交 body 已补(rationale + r1 处置摘要)。测试位置已移到 use super::* 之后。

验证:agent_turn 1/1、sampling 7/7、anthropic 71/71、kimi 124/124;fmt ✅。clippy -D warnings 的 6 条 too_many_arguments 为仓内既知存量(context_report/engine/turn_loop/lib.rs,与本 PR 文件零交集)。

堆叠:维持 #130 → 本 PR 的合并顺序;#130 的 MINOR 项有任何改动我会随之 rebase。

1 similar comment
@sky11023

sky11023 commented Nov 4, 2026

Copy link
Copy Markdown
Author

r1 处置(head 97dd00aef)

MAJOR(同义反复钉测试):已替换为行为级三路由断言 agent_turn_temperature_survives_vanilla_and_is_stripped_on_sampling_fixed_routes——构造抽成 agent_turn_message_request(生产唯一 agent-turn 构造点),普通路由经生产 build_chat_wire_body 断 wire body temperature==0.3;Kimi-Code 会员路由经生产 apply_kimi_code_fixed_sampling 断剥除;Anthropic 拒绝采样族经新抽的 apply_anthropic_sampling 断两个采样参数都不带(含非拒绝族对照臂)。通道全部走生产函数,零重抄。

这个测试当场抓到一个真 bug:f32 采样参数直扩 f64 后,wire 上是 0.30000001192092896(17 位尾数)——旧的同义反复测试永远抓不到。已在序列化接缝修复:新增 sampling_wire_value(经 f32 最短十进制表示中转),chat 与 anthropic 两处落 wire 统一走它;既有 anthropic 测试钉的期望值恰是伪影值(Some(f64::from(0.7f32))=0.699999988079071),已更新为干净 0.7 并注明原委。

MINOR:①LMM_TEMPERATURE 溯源与②过渡期无覆盖的接受声明已写入 PR 正文(见「评审追补」节);③测试注释的错误引用随测试重写消除;④提交 body 已补(rationale + r1 处置摘要)。测试位置已移到 use super::* 之后。

验证:agent_turn 1/1、sampling 7/7、anthropic 71/71、kimi 124/124;fmt ✅。clippy -D warnings 的 6 条 too_many_arguments 为仓内既知存量(context_report/engine/turn_loop/lib.rs,与本 PR 文件零交集)。

堆叠:维持 #130 → 本 PR 的合并顺序;#130 的 MINOR 项有任何改动我会随之 rebase。

l28756 added 2 commits November 5, 2026 10:27
Cloud-budget 45s header wait exists to fail fast on dead cloud hosts;
a local model (loopback/private) legitimately spends minutes before
response headers when it prefills a very long prompt cold — pinvou3
field data (TensorFold dual-GPU, 2026-10-29): ~25K-token agent prompt
sits silent ~30s at 854 tok/s first-pass. endpoint_is_local() routes
loopback/localhost/RFC1918/link-local/ULA base URLs to a 180s budget
across all three streaming adapters; explicit env override still wins
for both classes. Per-chunk idle stays 900s (already >= the 120s
local contract).

Signed-off-by: l28756 <li.jiawen@h3c.com>
Signed-off-by: l28756 <li.jiawen@h3c.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants