Repository navigation
feat: native Anthropic provider with prompt caching support - #78
Open
justinwilkin wants to merge 8 commits into
Open
justinwilkin wants to merge 8 commits into
justinwilkin wants to merge 8 commits into
Conversation
Add a Stability enum (Volatile/Session/Static) and WithStability setter on Message so providers can decide where a cached prefix can safely end.
Add Provider (openai/anthropic) and MaxTokens fields to LLMSection with validation. Extend intFromAny to accept string input since the settings overlay submits text fields as strings.
Add pkg/llm/anthropic implementing the LLM provider interface via the Anthropic Messages API, and pkg/llm/factory to construct a provider from settings (provider name, api key, model, base URL, max tokens), replacing the openai-specific factory. See ADR 0055.
…ttings Switch forge-headless, forge headless, and TUI provider reload to factory.NewProvider/BuildProvider instead of constructing the OpenAI provider directly, so the CLI and TUI can target Anthropic too. Add provider and max_tokens fields to the TUI settings overlay.
Mark summary/goal-batch messages generated by the context compaction strategies as StabilitySession so they don't invalidate a cached static prefix.
FormatToolSchema, GenerateXMLExample, and getToolsList previously relied on Go's non-deterministic map/slice iteration, which produced different prompt text across requests. Sort keys and tool names so the prompt stays byte-identical, enabling provider-side prompt caching.
…hing Add PromptBuilder.BuildSegments and buildSystemMessages to split the system prompt into a StabilityStatic segment (fixed for the process lifetime) and a StabilitySession segment (tool listing, guidance, which may change during a session). Rework BuildMessages to take system messages plus history and append ephemeral context (retrieved memories, error recovery) as trailing user messages instead of concatenating them into the system prompt, so the cached prefix stays byte-identical across turns. Extract retrieveMemories out of preparePrompt accordingly. See ADR 0056.
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Clearing llm.max_tokens via the TUI cannot currently reset the value to 0/provider-default due to empty-string handling in numeric parsing.
Once you've addressed the issues Copilot identified, you can request another Copilot review.
Pull request overview
This PR adds a native Anthropic Messages API provider and a shared provider factory, and restructures prompt assembly to support provider-side prompt caching via stability-tagged system segments and deterministic prompt text.
Changes:
- Introduces
types.Stabilityon messages and restructures prompt building to keep stable prefixes cacheable (sorted tool/schema output + segmented system messages + ephemeral trailing context). - Replaces the OpenAI-only provider construction with a shared
pkg/llm/factorythat can build either OpenAI-compatible or Anthropic providers, includingproviderandmax_tokensconfig support. - Adds the native Anthropic provider implementation (streaming, document analysis, and cache breakpoint placement) plus associated tests and docs/ADRs.
File summaries
| File | Description |
|---|---|
| pkg/types/message.go | Adds Stability enum + Message.Stability and WithStability chaining helper. |
| pkg/llm/openai/factory.go | Removes OpenAI-specific BuildProvider (replaced by shared factory). |
| pkg/llm/factory/factory.go | Adds provider-agnostic settings resolution and provider construction dispatch. |
| pkg/llm/factory/factory_test.go | Adds tests for factory precedence, provider selection, and max_tokens. |
| pkg/llm/anthropic/anthropic.go | Implements native Anthropic provider (streaming, document analysis, cache control). |
| pkg/llm/anthropic/anthropic_test.go | Adds unit tests for Anthropic provider behavior and cache control placement. |
| pkg/executor/tui/reload.go | Switches TUI hot-reload to factory-based provider creation. |
| pkg/executor/tui/overlay/settings.go | Exposes provider and max_tokens fields in settings UI + runtime display logic. |
| pkg/config/provider_test.go | Updates config provider build tests to use shared factory. |
| pkg/config/memory.go | Extends numeric parsing helper to support string input (used by settings overlay). |
| pkg/config/llm.go | Adds provider + max_tokens to LLM config with validation and accessors. |
| pkg/config/llm_test.go | Adds tests for provider defaulting/validation and max_tokens parsing/validation. |
| pkg/config/llm_override_test.go | Updates override tests to use factory BuildProvider. |
| pkg/agent/tools.go | Sorts visible tools by name for deterministic prompt tool listing. |
| pkg/agent/tools_test.go | Adds test asserting tool list sorting by name. |
| pkg/agent/prompts/xml_example.go | Makes XML example property iteration deterministic via sorted keys. |
| pkg/agent/prompts/prompts_test.go | Updates BuildMessages call sites and adds segment/ephemeral ordering tests. |
| pkg/agent/prompts/formatter.go | Sorts schema property iteration and adds shared sortedKeys helper. |
| pkg/agent/prompts/builder.go | Splits system prompt into static/session sections and updates BuildMessages API. |
| pkg/agent/prompts.go | Builds system prompt as string and as stability-tagged system messages; sorts custom tools list. |
| pkg/agent/iteration.go | Moves retrieved memories into ephemeral trailing messages and uses system segments. |
| pkg/agent/context/tool_call_strategy.go | Tags generated summaries as session-stable. |
| pkg/agent/context/tool_call_strategy_test.go | Asserts session stability tagging on summarized messages. |
| pkg/agent/context/threshold_strategy.go | Tags generated summaries as session-stable. |
| pkg/agent/context/threshold_strategy_test.go | Asserts session stability tagging on summarized messages. |
| pkg/agent/context/goal_batch_strategy.go | Tags goal batch message as session-stable. |
| pkg/agent/context/goal_batch_strategy_test.go | Asserts session stability tagging on goal batch messages. |
| go.mod | Adds Anthropic SDK and bumps/testify plus indirect deps. |
| go.sum | Adds checksums for new/updated dependencies. |
| docs/reference/configuration.md | Documents llm.provider and llm.max_tokens with examples and precedence. |
| docs/adr/README.md | Adds ADR index entries for new ADRs. |
| docs/adr/0056-prompt-caching-strategy.md | Adds ADR describing stability tagging and cache breakpoint strategy. |
| docs/adr/0055-anthropic-native-provider.md | Adds ADR describing provider selection and native Anthropic provider design. |
| cmd/forge/main.go | Switches CLI startup path to shared factory and updates flag/env help text. |
| cmd/forge/headless.go | Switches headless mode provider construction to shared factory. |
| cmd/forge-headless/main.go | Switches standalone headless binary to shared factory and removes custom precedence logic. |
Review details
- Files reviewed: 35/36 changed files
- Comments generated: 1
- Review effort level: Lite
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+92
to
+94
| case string: | ||
| i, err := strconv.Atoi(strings.TrimSpace(n)) | ||
| return i, err == nil |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a native Anthropic provider alongside a shared provider factory, and introduces message stability metadata so system prompts can be structured for provider-side prompt caching.
Changes
pkg/types/message.go): newStabilityenum (Volatile/Session/Static) andMessage.WithStabilityto mark how safely a message can sit in a cached prefix.pkg/config/llm.go,pkg/config/memory.go): newproviderandmax_tokensLLM settings, with validation and settings-overlay support.pkg/llm/anthropic/,pkg/llm/factory/): new native Anthropic provider implementation, replacingpkg/llm/openai/factory.gowith a sharedfactory.NewProvider/factory.BuildProviderusable by any provider. See ADR 0055.cmd/forge-headless,cmd/forge/headless.go,pkg/executor/tui/reload.go,pkg/executor/tui/overlay/settings.go): switched to the provider factory, exposedprovider/max_tokensin the TUI settings overlay.pkg/agent/prompts/formatter.go,pkg/agent/prompts/xml_example.go,pkg/agent/tools.go): sort map keys and tool names so prompt text is byte-identical across requests.pkg/agent/prompts/builder.go,pkg/agent/prompts.go,pkg/agent/iteration.go): split the system prompt into aStabilityStaticsegment (fixed for process lifetime) and aStabilitySessionsegment (tool listing/guidance). ReworkedBuildMessagesto append ephemeral context (retrieved memories, error recovery) as trailing user messages instead of concatenating into the system prompt, keeping the cached prefix stable across turns. See ADR 0056.pkg/agent/context/*_strategy.go): tag generated summary/goal-batch messages asStabilitySessionso they don't invalidate the static cached prefix.github.com/anthropics/anthropic-sdk-go, bumpedtestify.Docs
docs/reference/configuration.mdTesting