Repository navigation
📎 feat: Automatic Reading Policy (llmDeliveryPolicy) for 0.8.8 - #101
Merged
Merged
Conversation
fileConfig.llmDeliveryPolicy (classic|automatic) replaces preferFileSearch; settable globally, on endpoints.default or per endpoint. Under automatic, spreadsheets go to Run Code and the model is told their sandbox path. When attachments together exceed half the context room or the extracted-text limit, the largest leave the prompt for File Search or Run Code and are listed for the tool. A turn that cannot fit the context window is refused before sending. OpenAI-format PDF file parts are estimated by page. Default off. Adds the queued Run Code file note that upstream has and 0.8.8 lacks. Ticket: GENAI-891
2 tasks done
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the
|
Upload records metadata.pageCount/textChars for PDF message attachments on automatic-reading endpoints. estimateAttachmentTokens prices them pages*1600 + chars/2.5 (never below the size estimate), fitted to live Bedrock Converse measurements (40 pages/271,780 chars = 171,779 tokens; 10 pages/67,870 chars = 43,318) where the size estimate was 2-3.6x low. Classic endpoints are unchanged. Ticket: GENAI-891
1 task done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Unified upload sends every attachment with no chosen destination straight into the prompt. Two Sjúkra cases break on that: 15 PDFs (36 MB) on Claude Sonnet 5.5 overflow the 1M window (Bedrock answers "Input is too long", the user sees "Your messages do not fit in the current context window"), and a 50,000-row CSV fails with "This turn exceeds the configured extracted document text limit (9452368 > 1000000)" even though Run Code is on.
This adds
fileConfig.llmDeliveryPolicy: classic | automatic(the name proposed upstream in LibreChat-AI#17370), top level or per endpoint, defaultclassic, which changes nothing. Underautomatic:defaultLLMDeliveryPathoverrides andlegacyFileUploadUXare untouched. When the model's window is unknown, only the spreadsheet rule applies.It also prices OpenAI chat
fileand Responsesinput_filePDF parts by page instead of a flat 2,000 or 0 tokens.Replaces #100 (preferFileSearch). Two commits (the policy, then the PDF pricing fix from the live run); squash-merge to land it as one.
Type of change
Testing
Automated tests:
packages/data-providerresolver and file-config specs: 374 passed.packages/apiinitialize, attachments, client, delivery, encode, resources, fallback and documents specs: 433 passed;data-schemasfile schema spec: 1 passed. They cover the four cases (15 PDFs, the 9.4M-character CSV, a small Word letter, a photo), off-by-default, the refusal, tool-load failure, resent history and endpoint-dropped files.api/BaseClient, agents client and Files process specs: 541 passed.tsc --noEmit(data-provider, packages/api) and eslint: clean.hooks/reaper.spec.tsandhooks/executor.spec.tsfail on this machine, the same as on an unmodified checkout (process-signal timing), and are untouched.Live run: see "PDF pricing" below (Bedrock Converse accepted 5 × 2.4 MB and 6 PDFs in one request).
Screenshots / recordings
No user-facing change.
Risk / compatibility
fileLimit(default 10 for this endpoint) must allow them; that is tenant config.RAG_API_URL; a file that cannot be embedded fails that search.PDF pricing: pages + extracted text
The size-based estimate (base64 bytes / 75,000 × 2,000) undercounted text-dense PDFs 2–3.6× on a live run (Bedrock Converse,
claude-sonnet-5-5, 2026-10-30, synthetic text PDFs):Five 2.4 MB PDFs measured 859k of a 1M window, but were estimated at 430k and kept inline. They now estimate at 864k, and three move to File Search. Six documents in one Converse request were accepted, so no document-count cap is added.
llmDeliveryPolicy: automatic, upload recordsmetadata.pageCountandmetadata.textCharsfor PDF message attachments whose destination was inferred. They come from the parser's fallback text when it exists, otherwise from one pdf.js pass (≤ 15 MB). Classic endpoints are unchanged. This also gives the existingpageCounttelemetry a writer.max(size estimate, pages × 1,600 + chars / 2.5). The per-page and per-char terms match Anthropic's docs (each page is sent as its text plus an image; 1,500–3,000 tokens per page) and fit both samples within 1%. The two samples have similar text per page, so the split between the terms is approximate.client.ts) still prices by size. Scanned PDFs ([Bug]: PDF token estimate counts pages by file size — scanned PDFs overcount ~16× and fail with empty_messages LibreChat-AI/LibreChat#17473) are therefore not fixed here. That needsclient.tsto see the record, and is left for a follow-up.Ticket: GENAI-891