Repository navigation
Conversation
asto18089
left a comment
There was a problem hiding this comment.
First formal review — #132 feat(tui): default agent-turn temperature to 0.3 (stacked on #130)
Review of commit 9725d2cc (the second commit; the first is #130's cb577ba5, reviewed separately). One change requested — see MAJOR. Stacked note: this PR contains #130 in full. Merge order must be #130 → #132; any change to #130 (including its MINOR items) requires this branch to rebase, and after #130 merges a rebase would let this PR's diff show only the temperature change.
Per-file findings
crates/tui/src/acp_server.rs — temperature: None → Some(DEFAULT_AGENT_TEMPERATURE)
- Scope verified:
acp_server.rs:1995is the only production construction site of the agent-turnMessageRequest(other sites are translation, cache-warmup copy, reviewer, and tests). The reviewer/guardian path already pinsSome(0.0), so explicit sampling on agentic paths has precedent; translation/compaction keep server defaults. Scope is consistent. - Rejecting routes are protected, so
Some(0.3)cannot leak where sampling is forbidden: the Anthropic adapter guards withanthropic_model_rejects_sampling(opus-4-7/4-8, *-5 families, fable, mythos); the Chat adapter stripstemperature/top_pfor exact direct-Moonshot-K3 and Kimi-Code membership routes; Responses writes temperature only whenSome. Verified on base. - Justification: "servers treat absent temperature as 1.0" is accurate for OpenAI-compatible stacks (and their typical self-hosted reimplementations). The MTP/speculative-acceptance claim is attributed to an internal report I cannot verify from this repo — plausible, and clearly attributed, so acceptable.
- MINOR (unverifiable cross-reference):
LMM_TEMPERATUREappears nowhere in CodeWhale orpinvou3-app(grepped both). The comment's claim that the local-engine card knob sets the server-side default and that an explicit request value overrides it is standard API semantics, but the knob itself cannot be verified from this codebase. Please link the engine-side definition or the ratification record (issue/report) in the PR description so the "2026-11-02 拍板" is traceable. - MINOR (no escape hatch): once this lands, agent-turn sampling is hard-coded with no in-repo override until the promised user-level sampling config exists. Users who tuned sampling via engine-side defaults silently lose it on agent turns. The comment discloses this — good — but please confirm maintainers explicitly accept the interim state.
- nit: the commit body is empty apart from the sign-off; the rationale lives only in code comments. A two-line body would help
git logarchaeology.
crates/tui/src/acp_server.rs — agent_turn_default_temperature_is_the_ratified_value
- MAJOR:
assert_eq!(DEFAULT_AGENT_TEMPERATURE, 0.3)is a pure tautology — it asserts a constant equals a literal and provides zero regression protection (it can only fail if someone edits the constant, which any diff would show). Please replace it with a behavioral assertion, e.g. build the ACP agent-turn request (or capture the wire body via the existing wiremock harness inclient.rstests) and assertbody["temperature"] == 0.3for a vanilla route, plus the negative space: notemperatureon a sampling-rejecting route (anthropic_model_rejects_samplinghit) and on the Kimi-Code stripping route. That would also fix the comment's claim below. - MINOR: the test comment says the
Some(t) → body.temperatureserialization contract is covered by "client 侧 Kimi 固定采样用例" — those tests assert the opposite direction (Some(0.3)is stripped on that route). Pass-through is covered elsewhere (e.g. the wire assertion oftemperature == 0.25on the neighbor route); cite that instead. - nit: the test fn is inserted before
use super::*;inmod tests. Compiles fine, unconventional placement — move it below the imports.
CI
No workflow runs exist for head 9725d2cc (runs list, check runs, and combined status all empty — first-contribution approval gate; zero runs in status=waiting). Nothing for me to approve via the API at my permission level — maintainers, please approve/trigger the workflow run for both this PR and #130.
Conclusion
- MAJOR: tautological pinned test — replace with a wire-level assertion of the new default plus the rejecting-route negative cases.
- MINOR: LMM_TEMPERATURE citation unverifiable in-repo (link it); no interim user override for the hard-coded 0.3 (confirm acceptance); test-comment cites the wrong covering test; empty commit body.
- The behavior change itself is well-contained and correctly guarded on all wire paths. Requesting changes only for the test; once replaced and #130 has settled, this is ready. Merge order: #130 first, then this (rebased).
asto18089
left a comment
There was a problem hiding this comment.
Second review — rationale re-verification of the 0.3 default (2026-11-04)
The first review provisionally accepted the justification ("servers treat absent temperature as 1.0 … acceptable"). That line is superseded. The rationale was re-verified from primary sources — vendor API docs, official model cards and generation_config.json files, serving-stack source code, and the coding-agent ecosystem — via seven investigation lanes (four external, three in-repo). The premise of the change does not hold for either endpoint class this PR serves. The standing MAJOR (tautological test) from round 1 remains open.
What omitting temperature actually does (all verified 2026-11-03/04)
| Endpoint | Omitted → | Vendor recommendation | Explicit value |
|---|---|---|---|
| OpenAI (GPT-5.x / o-series) | 1.0 | 1.0 — the only supported value ("Only the default (1) value is supported") | non-1.0 → 400 on reasoning models |
Anthropic Messages (2026-10-21) |
1.0 | 1.0 — temperature/top_p/top_k removed: sending any of them "returns a 400 error, even at the parameter's default value"; omitted requests "sample with their defaults, including a temperature of 1.0" |
deprecated → 400 |
| Google Gemini 3 | 1.0 | 1.0 — "we strongly recommend keeping the temperature parameter at its default value of 1.0"; below 1.0 "may lead to unexpected behavior, such as looping or degraded performance"; migration guide says to remove explicit low values | warned against |
| DeepSeek API | 1.0 | V3.2 (final) ships generation_config.json temperature: 1.0, top_p: 0.95; SGLang's official V3.2 recipe: "temperature = 1.0, top_p = 0.95 (as recommended by DeepSeek)"; temperature "has no effect in thinking mode". (Legacy task table recommends 0.0 for coding/math — pre-thinking-era guidance for old non-thinking models) |
accepted |
| Qwen3 (DashScope) | 0.7 non-thinking / 0.6 thinking | = card recommendations (thinking 0.6/0.95, non-thinking 0.7/0.8); "DO NOT use greedy decoding" | accepted |
| Zhipu GLM-4.6 | 1.0 | 1.0 ("we recommend using a sampling temperature of 1.0", + top_p 0.95 / top_k 40) | accepted |
| Moonshot Kimi (k2.6+/K3/k2.7-code) | n/a — parameter absent from the current OpenAPI spec | fixed server-side (1.0 thinking/code, 0.6 k2.6 non-thinking); "Do not pass temperature explicitly when calling these models" | error on any explicit value |
| MiniMax M2/M2.1 | recommended 1.0 | 1.0 / top_p 0.95 / top_k 40; M3 ignores explicit values entirely | ignored |
| vLLM ≥ 0.8.0 | model's generation_config.json (--generation-config auto is the default since vllm#12622); 1.0 is only the last-resort fallback |
the model author's shipped values | client value overrides generation config |
| SGLang ≥ 0.5.4 | model generation config (--sampling-defaults model, the default); 1.0 fallback |
same | client value wins |
| Ollama (Sep 2026+) / llama.cpp (Nov 2025+) | GGUF-embedded / generation_config defaults; 0.8 only when the model ships none | same | client value wins |
Sources: Anthropic migration 2026-10-21, Gemini 3 guide, DeepSeek parameter settings, DashScope defaults, GLM-4.6 card, Kimi models overview, MiniMax-M2.1 card, vLLM OpenAI-compat docs, SGLang sampling params, Ollama PR #16471, llama.cpp PR #17120.
Why the 0.3 rationale fails on both endpoint classes
- Cloud: "omitted → 1.0" is true, but 1.0 is the vendors' recommendation for current flagships — not a pathology to be corrected client-side. Gemini explicitly warns that sub-1.0 values degrade reasoning tasks; Anthropic removed the knob entirely; OpenAI reasoning models reject non-1.0. For older Anthropic models on unguarded routes, the PR now sends 0.3 where the server previously applied the per-model default — exactly what the Gemini migration guide tells integrators to stop doing.
- Local: "most inference servers treat missing temperature as 1.0" is not the current behavior of the stacks this PR's own first commit targets. vLLM ≥ 0.8.0, SGLang ≥ 0.5.4, and current Ollama/llama.cpp all prefer the model's shipped sampling defaults (that is what
generation_config.jsonis for); 1.0 is only a last-resort fallback. An explicit client value overrides those vendor recommendations on every stack — soSome(0.3)forces, e.g., a correctly-served Qwen3-thinking from its recommended 0.6 down to 0.3, and DeepSeek-V3.2-Exp from 0.6 down to 0.3. The first commit of this very PR treats local endpoints as first-class; the second commit overrides their tuned defaults. - No vendor recommends 0.3. The recommendation landscape is: 1.0 for the current flagship generation (OpenAI, Anthropic, Gemini, DeepSeek V3.2, GLM-4.6, Kimi thinking/code, MiniMax), or 0.6–0.7 for the reasoning-era minority (Qwen3, DeepSeek-Exp/V3.1/R1, Kimi K2-Instruct). 0.3 matches no published recommendation on any route it reaches.
What survives scrutiny
- The MTP direction has one public corroboration: llama.cpp documents that with rejection-sampling verification "acceptance falls away as temperature rises" (PR #27694). But vLLM/SGLang MTP drafters are greedy by default (temperature-independent), and no vendor or stack publishes "keep temperature at X for accept rate" guidance — DeepSeek's own V3.2 recipe is 1.0/0.95 despite MTP. The cited TensorFold 10-30 report remains unverifiable from outside.
- The wire guards hold: verified that
Some(0.3)is stripped on Kimi-Code/direct-K3 membership routes (client/chat.rs:923-968) andanthropic_model_rejects_samplingroutes (client/anthropic.rs:734-748); it reaches the wire on plain Chat Completions and older-Anthropic routes, and on Responses routes only for DeepSeek (client/responses.rs:140-147).
In-repo evidence against a hard-coded client default
- This codebase already lived the failure mode: compaction once hard-coded 0.3, it leaked onto the Kimi Code membership route and failed the whole compaction pass; the fix removed the fixed value (compaction now sends
None, pinned by tests atcompaction.rs:2329/2660). PR #132 reintroduces the same pattern one path over. - Scope inconsistency: commit
9725d2cc0touches only the ACP server path. The interactive TUI's own main turn (crates/core/src/request.rs:76), subagents, RLM, one-shot exec, and MCP chat all still sendNone— so the ratified "agent turns need stable output" premise is not applied to the product's own agent loop, only to external ACP clients. LMM_TEMPERATUREis unverifiable: zero hits in both CodeWhale andpinvou3-app(grepped this round). The comment's override interaction is standard semantics (client value always wins), but the knob itself is not in any repo this PR touches — please link the engine-side definition or the ratification record. And since no user/model-level sampling config exists in-repo (no config field, no env var), any engine-side default is silently overridden with no escape hatch.- App-side prior art mandates omission: the pinvou3 settings-layer image probe deliberately omits temperature — "kimi-for-coding 等模型只接受默认值,显式传会 400" (
settings.rs:811, test-pinned) — consistent with Kimi's current "do not pass explicitly" contract. - Ecosystem direction supports omission over hardcoding: Codex CLI omits temperature, OpenAI Agents SDK defaults to
None, LangChain moved 0.7 → omitted, Claude Code has no sampling knob, and Gemini CLI is the notable (aging) exception that still sends explicit 0 on its main loop.
Requested changes
- Primary: revert
temperature: Some(DEFAULT_AGENT_TEMPERATURE)back toNone. Omitting is the only choice that yields the vendor-recommended value on every verified endpoint — cloud and local — and it matches where the API ecosystem is heading (parameter removal/rejection). If the "2026-11-02 拍板" decision survives this evidence, it needs re-ratification by maintainers with these sources on record. - Alternative: if a deterministic agent-turn default is genuinely wanted, implement it as an explicit opt-in (model-level config field, the already-promised user-level sampling config), not a silent hardcode — and do not pick 0.3, which sits off every vendor's recommendation.
- Unchanged from round 1: replace the tautological constant test with wire-level assertions (0.3 present on a vanilla route; absent on the Anthropic-rejecting and Kimi stripping routes), fix the test comment's wrong cross-reference, and move the test below
use super::*. - Stacking: #130 has advanced to
ca079796cwhile this PR still carries the oldcb577ba5— after #130 merges, rebase so the diff shows only the temperature change. CI remains empty on9725d2cc0(first-contribution gate) — maintainers, please still approve/trigger the workflow runs for #130 and #132.
9725d2c to
ad1b8ad
Compare
r1 处置(head 97dd00aef)MAJOR(同义反复钉测试):已替换为行为级三路由断言 这个测试当场抓到一个真 bug:f32 采样参数直扩 f64 后,wire 上是 MINOR:①LMM_TEMPERATURE 溯源与②过渡期无覆盖的接受声明已写入 PR 正文(见「评审追补」节);③测试注释的错误引用随测试重写消除;④提交 body 已补(rationale + r1 处置摘要)。测试位置已移到 验证:agent_turn 1/1、sampling 7/7、anthropic 71/71、kimi 124/124;fmt ✅。clippy -D warnings 的 6 条 too_many_arguments 为仓内既知存量(context_report/engine/turn_loop/lib.rs,与本 PR 文件零交集)。 |
1 similar comment
r1 处置(head 97dd00aef)MAJOR(同义反复钉测试):已替换为行为级三路由断言 这个测试当场抓到一个真 bug:f32 采样参数直扩 f64 后,wire 上是 MINOR:①LMM_TEMPERATURE 溯源与②过渡期无覆盖的接受声明已写入 PR 正文(见「评审追补」节);③测试注释的错误引用随测试重写消除;④提交 body 已补(rationale + r1 处置摘要)。测试位置已移到 验证:agent_turn 1/1、sampling 7/7、anthropic 71/71、kimi 124/124;fmt ✅。clippy -D warnings 的 6 条 too_many_arguments 为仓内既知存量(context_report/engine/turn_loop/lib.rs,与本 PR 文件零交集)。 |
Cloud-budget 45s header wait exists to fail fast on dead cloud hosts; a local model (loopback/private) legitimately spends minutes before response headers when it prefills a very long prompt cold — pinvou3 field data (TensorFold dual-GPU, 2026-10-29): ~25K-token agent prompt sits silent ~30s at 854 tok/s first-pass. endpoint_is_local() routes loopback/localhost/RFC1918/link-local/ULA base URLs to a 180s budget across all three streaming adapters; explicit env override still wins for both classes. Per-chunk idle stays 900s (already >= the 120s local contract). Signed-off-by: l28756 <li.jiawen@h3c.com>
Signed-off-by: l28756 <li.jiawen@h3c.com>
ad1b8ad to
4a5dd20
Compare
问题
Agent 回合的
MessageRequest从不显式下发采样参数,多数推理服务端按temperature=1.0处理——随机性最大,且实测会略降 MTP 类引擎的投机接受率(TensorFold 10-30 性能报告②:默认 1.0 vs 低温的接受率差异)。修复(2026-11-02 拍板)
acp_server.rs的 Agent 回合请求构造处:temperature: None→Some(DEFAULT_AGENT_TEMPERATURE),拍板值 0.3(兼顾确定性与多样性),以具名常量+钉测试防漂移。apply_kimi_code_fixed_sampling等按 provider 剥离 temperature 的既有逻辑在序列化之后仍生效。LMM_TEMPERATURE旋钮改的是服务端缺省,显式请求值会覆盖它——需要引擎侧自定义采样时应在模型配置层显式设置(未来引入用户级采样配置后由用户值优先)。验证
cargo check -p codewhale-tui绿;cargo fmt --check绿;clippy 无新增(基线存在与本改动无关的历史 too_many_arguments 项)。agent_turn_default_temperature_is_the_ratified_value通过;序列化契约(Some(t)→body.temperature)由 client 侧既有 Kimi 固定采样用例覆盖。cb577ba55(= PR fix(tui): widen local-endpoint SSE open timeout to 180s #130 的 180s 超时修复,独立合并互不依赖)。DCO:Signed-off-by 已带。
评审追补(r1 处置说明,见评论区)
LMM_TEMPERATURE是品悟模型商店卡片脚本侧的引擎旋钮(随部署注入推理服务环境,定义见品悟仓 docs/模型卡片脚本规范.md §环境变量;本仓库不携带该规范,属跨仓引用)。2026-11-02 拍板记录在品悟侧任务书(双机性能优化②:品悟默认下发 temperature=0.x)。