docs: record Data's model and the AI Gateway provider - #20
EthanThatOneKid wants to merge 2 commits into
Conversation
Data's inference now runs through an openai-compatible provider pointed at the Vercel AI Gateway, so turns are metered to the account's credits instead of a separate provider key. Record why the local backend's own vercel-ai-gateway/* handles cannot be used (Anthropic Messages API requires max_tokens, and the catalog carries no max-output field, so every turn fails with 'max_tokens: Invalid input: expected number, received null').
|
This pull request updates the documentation to record Data's current model and AI Gateway provider. The change addresses past issues with provider capacity by switching to a Vercel AI Gateway model,
Review the added documentation for accuracy and clarity regarding the model and provider configuration. |
|
This pull request updates the documentation to record Data's current model and AI Gateway provider. The change addresses past issues with provider capacity by switching to a Vercel AI Gateway model,
Review the added documentation for accuracy and clarity regarding the model and provider configuration. |
|
Superseded by #22. This documents the Letta model and provider setup; that whole configuration moved to |
Data's inference was failing on provider capacity, not on a broken service: the OpenRouter free pool's daily cap and Google's per-minute free-tier limit each surfaced as a 502 from
/ask, with no fallback.The agent now runs
openai-compatible/openai/gpt-5-minithrough a Vercel AI Gateway provider registered athttps://ai-gateway.vercel.sh/v1, so turns are metered to the account's AI credits rather than a separate provider key.Recorded, because the obvious route does not work:
vercel-ai-gateway/*handles fail on every turn with400 max_tokens: Invalid input: expected number, received null— that provider speaks the Anthropic Messages API, wheremax_tokensis required, and its catalog entries carry a context window but no max-output field. Reproduced on this host before switching.openai-compatibleroute setsmax_tokensitself (32000here), and the same gateway key works through it.~/.letta/lc-local-backend/providers/), so the service needs no gateway key in its environment.Verified live: a direct
POST /v1/chat/completionsto the gateway returned200; a headless Letta turn on the agent answered; andPOST /askon the restarteddata-httpreturned200with an answer. No credential value is recorded in either file.