Skip to content

perf: drop always-on tools from the fixed prompt - #463

Closed
hetaoBackend wants to merge 1 commit into
mainfrom
perf/trim-fixed-prompt-tools
Closed

hetaoBackend wants to merge 1 commit into
mainfrom
perf/trim-fixed-prompt-tools

Conversation

@hetaoBackend

@hetaoBackend hetaoBackend commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

Problem

A review of the fixed prompt (everything sent on every request before the user says anything) found tools and skill text that most sessions never use:

  • website_deploy (~867 tok) is registered unconditionally, plus the deploy-website / edit-deployed-website skills (~121 tok in the catalog). The tool publishes through the managed MiniMax account gateway, so it cannot work without an account session (API-key and custom-provider users have no access token).
  • update_goal (~769 tok) and get_goal (~55 tok) are sent whenever the thread-goal feature is on, even if the session has no goal. The model never creates goals (/goal and REST do), so with no goal it has nothing to read or update.
  • Skill descriptions are capped at 1024 code points each; 9 of 17 builtin skills exceed 300.

Changes

  1. website_deploy: buildLocalNativeRuntimeTools creates the deploy adapter only when authContext.accessToken is non-blank (new hasManagedAccountSession). deploy-website / edit-deployed-website are removed from resolveFeatureAwareBuiltinSkillNames via a new websiteDeployAvailable gate, passed by both callers (local-native-tools.ts, hosted-agent-skill-capabilities.ts), so they drop out of the skill catalog and the skill tool together with the tool. Callers that don't pass the gate behave as before.
  2. Goal tools: GoalLifecycle.runtimeToolsFor now uses sessionId: no goal record -> no tools; complete -> get_goal only; any other status (active, paused, budget_limited, blocked, usage_limited) -> both. The /goal kickoff turn already has the goal record when its tools are built, so it still gets both.
  3. Skills catalog: LOCAL_SKILL_DESCRIPTION_MAX_CODE_POINTS 1024 -> 300. Total cap (5000) and protected skills (mcode-tools-master, minimax-code-product) unchanged. Builtin descriptions are not rewritten.

Does the API-key / custom-provider path have an authContext? In the TUI (packages/tui/src/runtime/lifecycle.ts, launcher.ts) authContext.accessToken is only populated from the MiniMax OAuth/CLI account login; BYOK providers resolve credentials in the model resolver, not through authContext. So API-key-only users now get no website_deploy.

Measured savings

Scenario: default resolveAgentCapabilities() (all builtin tools), agent mavis, no MCP servers, 17 builtin skills. Tokens are counted with the repo's gpt-tokenizer (OpenAI cl100k-style, NOT MiniMax's tokenizer, so read the numbers as relative). Tool size = JSON {type:function, function:{name, description, parameters}}. Both trees were measured with the same script (main c172cf9 vs this branch).

Scenario Tools before -> after Skills catalog before -> after Total saved
API key, no goal 7987 -> 6296 (17 -> 14 tools) 1610 -> 1065 (16 -> 14 skills) ~2236
Logged in, no goal 7987 -> 7163 (17 -> 15) 1610 -> 1186 (16 skills) ~1248
Logged in, active goal 7987 -> 7987 (same as above) ~424 (catalog only)
Logged in, completed goal 7987 -> 7218 (17 -> 16) (same as above) ~769 + ~424

This is below the 2.5k-3.5k first estimate for logged-in users; API-key users get ~2.2k. These tokens sit at the start of the request and are normally served from the prefix cache, so the real gain is cheaper cached reads and more context headroom (also a smaller first/uncached request and a smaller body for the 413 / body-limit problem), not fresh input tokens saved on every turn.

Tests

  • New: goal-lifecycle-runtime-tools.test.ts (no goal / complete / each unfinished status / disabled), feature-owned-skills-website-deploy.test.ts (gate + hasManagedAccountSession), skills-catalog-description-cap.test.ts (300 cap, protected skill untouched), and a website_deploy auth matrix in local-native-tools.test.ts (no auth / no token / blank token / token).
  • Updated host-integration-settlement.test.ts: update_goal is no longer exposed for a complete goal, so that case asserts the tool list instead of the runtime rejection (the active-goal rejection case is kept).
  • New test files are registered in test/vitest-suites.json and release/public-source.json (other open PRs also edit these; expect add-line conflicts).
  • Node 22.20.0: node scripts/run-vitest-suite.mjs capability -> 217 files, 5404 passed, 18 skipped; pnpm typecheck and pnpm lint clean. Not run: check:standalone, byok/smoke (need a build; left to CI).

Risks

  • Skill hit rate: 300 chars cuts descriptions of lark-tools (1578), deep-research (948), visual-page (565), init (472), skill-creator, code-review, deploy-website mid-text. Trigger phrases placed late are lost. Mitigation would be rewriting those builtin descriptions with the trigger up front; intentionally not done here.
  • Skills/tools are fixed per turn: a goal created via /goal while a turn is running only gets tools on the next turn. A session whose login lapses loses website_deploy from the next turn.
  • Blocked and usage_limited goals keep update_goal (can still be resumed/edited); only complete is treated as finished.

Open questions for Tao

  1. Is 300 the right per-description cap, or should the builtin descriptions be rewritten first (and kept ~<=300)?
  2. Should blocked / usage_limited goals also be get_goal-only?
  3. Is "MiniMax account login" the right condition for website_deploy, or do some non-login hosts (e.g. Electron) need it via an explicit config flag?
  4. Savings need re-measuring with MiniMax's own tokenizer.

Draft, not ready for review; no reviewers requested.


View with [code]smith Autofix with [code]smith
Need help on this PR? Tag @codesmith-bot with what you need. Autofix is disabled.

- Only wire website_deploy (and its deploy-website / edit-deployed-website
  skills) when a MiniMax account access token is present.
- Only expose goal tools when the session has a goal: none for no goal,
  get_goal only once complete, both otherwise.
- Cap per-skill catalog descriptions at 300 code points (was 1024); the
  5000-token total cap and protected skills are unchanged.
@hetaoBackend
hetaoBackend marked this pull request as ready for review October 9, 2026 06:16
@hetaoBackend

Copy link
Copy Markdown
Collaborator Author

Closing for now: changing the tool list mid-session (goal tools appearing/disappearing, website_deploy on login changes) would invalidate the prompt cache prefix, which costs more than the tokens saved. We can revisit the parts that are stable for the whole session (e.g. the skills description cap) separately.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant