Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -404,6 +404,7 @@ Kit used the fewest tokens, took the least active time, and needed the least ste
- [Agent Plugins](docs/user/agent-plugins.md)
- [TUI and sessions](docs/user/tui-and-sessions.md)
- [Provider usage](docs/user/provider-usage.md)
- [Workload identity for inference](docs/user/workload-inference.md)
- [Security, limits, and troubleshooting](docs/user/security-limits-and-troubleshooting.md)
- [Releasing](docs/releasing.md) · [Third-party notices](THIRD_PARTY_NOTICES.md)

Expand Down
6 changes: 6 additions & 0 deletions docs/user/getting-started-and-configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,12 @@ kit prompt --provider openrouter \

Kit uses the CLI or TOML `model`; `OPENROUTER_MODEL` does not override that selection. The adapter also accepts `OPENROUTER_BASE_URL`, `OPENROUTER_APP_NAME`, `OPENROUTER_SITE_URL`, `OPENROUTER_MAX_COMPLETION_TOKENS`, `OPENROUTER_TEMPERATURE`, and `OPENROUTER_REASONING_EFFORT`. Its model-catalog lookup for the selected model's context length is best-effort, so a catalog failure does not by itself prevent normal provider usage.

### Workload identity for inference
Comment thread
danielkov marked this conversation as resolved.

For deployed workloads with projected identity tokens, use
[workload identity for inference](workload-inference.md) instead of local API-key
or browser authentication.

### TypeSafe API key

TypeSafe authentication is BYOK (bring your own key). Create a key at
Expand Down
73 changes: 73 additions & 0 deletions docs/user/workload-inference.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,73 @@
# Workload identity for inference

Use workload identity for deployed workloads that receive projected identity tokens.
For interactive local use, follow [provider authentication](getting-started-and-configuration.md#choose-a-provider-and-authenticate) instead.

For `openrouter` or `speakeasy`, configure a provider-specific workload exchange
in `~/.kit/config.toml`. Kit reads a deployment-projected JWT file; it does not
mint the external assertion. The deployment must rotate that file before renewal.
These settings override API-key and browser credentials for that provider only.
An exchange failure stops inference: Kit never falls back to another credential.

OpenRouter uses [workload identity federation](https://openrouter.ai/docs/guides/overview/auth/workload-identity-federation)
with an organization federation policy targeting an inference-capable key:

```toml
provider = "openrouter"
model = "anthropic/claude-sonnet-4"

[inference_providers.openrouter]
endpoint = "https://openrouter.ai/api/v1/chat/completions"
[inference_providers.openrouter.workload]
tokenFile = "/var/run/identity/assertion.jwt"
endpointMode = "explicit"
grant = "tokenExchange"
tokenEndpoint = "https://openrouter.ai/api/v1/oauth/token"
scope = "inference"
[inference_providers.openrouter.workload.extensions]
federation_policy_id = "<FEDERATION_POLICY_ID>"
```

Speakeasy uses its shared workload issuer's JWT-bearer token endpoint. Configure
issuer trust and project authorization on the server first, then use the actual
token endpoint returned for that issuer:

```toml
provider = "speakeasy"
model = "anthropic/claude-sonnet-4"

[inference_providers.speakeasy]
endpoint = "https://ai.speakeasy.com/chat/completions"
[inference_providers.speakeasy.workload]
tokenFile = "/var/run/identity/assertion.jwt"
endpointMode = "explicit"
grant = "jwtBearer"
tokenEndpoint = "https://ai.speakeasy.com/oauth/usi/<ISSUER_ID>/token"
resource = "urn:speakeasy:inference"
[inference_providers.speakeasy.workload.extensions]
project_id = "<PROJECT_ID>"
```

The Speakeasy assertion's audience is the shared issuer URL or its token endpoint,
not `urn:speakeasy:inference`. Assertions with a `jti` (or `uti`) are single-use: a new
exchange needs a fresh assertion. This is ordinary workload inference, not a
replacement for managed assistant credentials or chat-bound authorization. Kit
sends bearer authentication without API-key, project, or assistant/chat headers.

Both examples pin HTTPS token and completion endpoints explicitly; redirects are
disabled. Optional `audience`, `resource`, `scope`, and string-valued `extensions`
belong to the workload exchange, not completion headers. `endpointMode = "discovery"`
instead requires `expectedIssuer`, same-origin issuer metadata, and an HTTPS
`resource`; use explicit mode for Speakeasy's inference URN. Keep this configuration
operator-controlled: the endpoints determine where credentials are sent.

Kit shares in-memory tokens across sessions for each configured provider, caches
only until the returned `expires_in` minus a safety margin, and rereads the source
file on renewal. An RFC 8693 response without `expires_in` is used once and not
cached. A token may outlive its source assertion when the server permits it.
Tokens and assertions are not saved in Kit's credential store. Workload requests
are not automatically replayed on authentication refusal, throttling, or server
errors; API-key retry behavior is unchanged. Select models explicitly: workload
grants do not imply model-catalog access, and Kit does not discover a context limit
through these grants. Other features with separate credentials (such as images or
evaluation) are not authenticated by this configuration.
34 changes: 34 additions & 0 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1382,6 +1382,8 @@ struct Config {
root: Option<PathBuf>,
model: Option<String>,
provider: Option<kit::ProviderKind>,
#[serde(default)]
inference_providers: BTreeMap<String, kit::provider::inference::Config>,
reasoning_effort: Option<kit::ReasoningEffort>,
a2a: Option<String>,
capture_error_spans: Option<bool>,
Expand Down Expand Up @@ -2305,6 +2307,10 @@ async fn run_cli(cli: Cli) -> Result<(), Box<dyn std::error::Error>> {
openrouter_api_key.as_ref().map(|(key, _)| key.clone()),
)?,
};
let runtime = kit::Runtime::with_inference_providers(
runtime,
config.inference_providers.clone(),
)?;
let runtime = kit::Runtime::with_compaction(runtime, config.compaction)?;
let runtime = kit::Runtime::with_eval(runtime, config.experimental.eval)?;
let runtime = kit::Runtime::with_ask_user(runtime, config.ask_user.unwrap_or(true))?;
Expand Down Expand Up @@ -2406,6 +2412,10 @@ async fn run_cli(cli: Cli) -> Result<(), Box<dyn std::error::Error>> {
openrouter_api_key.as_ref().map(|(key, _)| key.clone()),
)?,
};
let runtime = kit::Runtime::with_inference_providers(
runtime,
config.inference_providers.clone(),
)?;
let runtime = kit::Runtime::with_compaction(runtime, config.compaction)?;
let runtime = kit::Runtime::with_eval(runtime, config.experimental.eval)?;
let runtime = kit::Runtime::with_ask_user(runtime, config.ask_user.unwrap_or(true))?;
Expand Down Expand Up @@ -2482,6 +2492,10 @@ async fn run_cli(cli: Cli) -> Result<(), Box<dyn std::error::Error>> {
reasoning_effort,
openrouter_api_key.as_ref().map(|(key, _)| key.clone()),
)?;
let runtime = kit::Runtime::with_inference_providers(
runtime,
config.inference_providers.clone(),
)?;
let runtime = kit::Runtime::with_compaction(runtime, config.compaction)?;
let runtime = kit::Runtime::with_eval(runtime, config.experimental.eval)?;
let runtime = kit::Runtime::with_ask_user(runtime, config.ask_user.unwrap_or(true))?;
Expand Down Expand Up @@ -4604,6 +4618,26 @@ future_option = true
let _rebound = tokio::net::TcpListener::bind(address).await.unwrap();
}

#[test]
fn inference_provider_configuration_parses_documented_examples() {
let docs = include_str!("../docs/user/workload-inference.md");
let mut count = 0;
for block in docs.split("```toml\n").skip(1) {
let example = block.split("```").next().unwrap();
if !example.contains("[inference_providers.") {
continue;
}
let config: Config = toml::from_str(example).unwrap();
assert_eq!(config.inference_providers.len(), 1);
let provider = config.provider.unwrap();
let inference = config.inference_providers.get(provider.as_str()).unwrap();
inference.workload.validate().unwrap();
assert!(inference.endpoint.ends_with("/chat/completions"));
count += 1;
}
assert_eq!(count, 2);
}

#[test]
fn subagent_model_policies_are_scoped_and_validated_per_harness() {
let configured: Config = toml::from_str(
Expand Down
Loading
Loading