Skip to content

📎 feat: Automatic Reading Policy (llmDeliveryPolicy) for 0.8.8 - #101

Merged
TomasPalsson merged 2 commits into
aproorg/0.8.8from
feat/automatic-reading-0.8.8
Oct 30, 2026
Merged

TomasPalsson merged 2 commits into
aproorg/0.8.8from
feat/automatic-reading-0.8.8

Conversation

@TomasPalsson

@TomasPalsson TomasPalsson commented Oct 30, 2026 •

Copy link
Copy Markdown

Summary

Unified upload sends every attachment with no chosen destination straight into the prompt. Two Sjúkra cases break on that: 15 PDFs (36 MB) on Claude Sonnet 5.5 overflow the 1M window (Bedrock answers "Input is too long", the user sees "Your messages do not fit in the current context window"), and a 50,000-row CSV fails with "This turn exceeds the configured extracted document text limit (9452368 > 1000000)" even though Run Code is on.

This adds fileConfig.llmDeliveryPolicy: classic | automatic (the name proposed upstream in LibreChat-AI#17370), top level or per endpoint, default classic, which changes nothing. Under automatic:

  • Spreadsheets (csv, tsv, xls, xlsx, ods, parquet) go to Run Code when the turn runs it, and the model is told the sandbox path (0.8.8 lacks upstream's queued-file note, so this adds it).
  • The other attachments stay in the prompt while together they fit half of the model's context room (window minus output) and the extracted-text limit. Past that, the largest leave first for File Search or Run Code, and File Search files are listed for the model so the first search indexes them.
  • A turn whose remaining inline attachments cannot fit the window at all is refused before anything is sent, with an "attachment context (estimated tokens)" limit error.
  • Media, chosen destinations, the agent's own files, per-type defaultLLMDeliveryPath overrides and legacyFileUploadUX are untouched. When the model's window is unknown, only the spreadsheet rule applies.

It also prices OpenAI chat file and Responses input_file PDF parts by page instead of a flat 2,000 or 0 tokens.

Replaces #100 (preferFileSearch). Two commits (the policy, then the PDF pricing fix from the live run); squash-merge to land it as one.

Type of change

  • Bug fix
  • Feature

Testing

Automated tests:

  • packages/data-provider resolver and file-config specs: 374 passed.
  • packages/api initialize, attachments, client, delivery, encode, resources, fallback and documents specs: 433 passed; data-schemas file schema spec: 1 passed. They cover the four cases (15 PDFs, the 9.4M-character CSV, a small Word letter, a photo), off-by-default, the refusal, tool-load failure, resent history and endpoint-dropped files.
  • api/ BaseClient, agents client and Files process specs: 541 passed.
  • tsc --noEmit (data-provider, packages/api) and eslint: clean.
  • Two reviewers ran mutation checks on every rule; each mutation now fails a test.
  • hooks/reaper.spec.ts and hooks/executor.spec.ts fail on this machine, the same as on an unmodified checkout (process-signal timing), and are untouched.

Live run: see "PDF pricing" below (Bedrock Converse accepted 5 × 2.4 MB and 6 PDFs in one request).

Screenshots / recordings

No user-facing change.

Risk / compatibility

  • Off by default.
  • With 15+ attachments the composer's fileLimit (default 10 for this endpoint) must allow them; that is tenant config.
  • File Search needs RAG_API_URL; a file that cannot be embedded fails that search.

PDF pricing: pages + extracted text

The size-based estimate (base64 bytes / 75,000 × 2,000) undercounted text-dense PDFs 2–3.6× on a live run (Bedrock Converse, claude-sonnet-5-5, 2026-10-30, synthetic text PDFs):

file pages extracted chars Bedrock input tokens / file old estimate new estimate
2.4 MB 40 271,780 171,779 (5 files: 858,895) 86,000 172,712 (+0.5%)
0.3 MB 10 67,870 43,318 (6 files: 259,906) 12,000 43,148 (−0.4%)

Five 2.4 MB PDFs measured 859k of a 1M window, but were estimated at 430k and kept inline. They now estimate at 864k, and three move to File Search. Six documents in one Converse request were accepted, so no document-count cap is added.

  • On endpoints with llmDeliveryPolicy: automatic, upload records metadata.pageCount and metadata.textChars for PDF message attachments whose destination was inferred. They come from the parser's fallback text when it exists, otherwise from one pdf.js pass (≤ 15 MB). Classic endpoints are unchanged. This also gives the existing pageCount telemetry a writer.
  • A provider-route PDF with those numbers costs max(size estimate, pages × 1,600 + chars / 2.5). The per-page and per-char terms match Anthropic's docs (each page is sent as its text plus an image; 1,500–3,000 tokens per page) and fit both samples within 1%. The two samples have similar text per page, so the split between the terms is approximate.
  • The estimate never goes below the size estimate, because the encoded-block count before pruning (client.ts) still prices by size. Scanned PDFs ([Bug]: PDF token estimate counts pages by file size — scanned PDFs overcount ~16× and fail with empty_messages LibreChat-AI/LibreChat#17473) are therefore not fixed here. That needs client.ts to see the record, and is left for a follow-up.
  • Records uploaded before this change, PDFs over 15 MB, and PDFs uploaded on classic endpoints keep today's estimate. Existing tests are unchanged.
  • The constants are calibrated on Claude's visual PDF mode. Providers that bill less per page (Gemini, Bedrock text-only document chat) are overestimated, which can only divert earlier.

Ticket: GENAI-891

fileConfig.llmDeliveryPolicy (classic|automatic) replaces preferFileSearch; settable globally, on endpoints.default or per endpoint.
Under automatic, spreadsheets go to Run Code and the model is told their sandbox path.
When attachments together exceed half the context room or the extracted-text limit, the largest leave the prompt for File Search or Run Code and are listed for the tool.
A turn that cannot fit the context window is refused before sending.
OpenAI-format PDF file parts are estimated by page. Default off.
Adds the queued Run Code file note that upstream has and 0.8.8 lacks.

Ticket: GENAI-891
@coderabbitai

coderabbitai Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

Run configuration
  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: Organization UI
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 59bee220-a36d-4565-bed1-496cac4cfb70

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Comment @coderabbitai help to get the list of available commands.

Upload records metadata.pageCount/textChars for PDF message attachments on automatic-reading endpoints.
estimateAttachmentTokens prices them pages*1600 + chars/2.5 (never below the size estimate), fitted to
live Bedrock Converse measurements (40 pages/271,780 chars = 171,779 tokens; 10 pages/67,870 chars =
43,318) where the size estimate was 2-3.6x low. Classic endpoints are unchanged.

Ticket: GENAI-891
@TomasPalsson
TomasPalsson merged commit ea54db7 into aproorg/0.8.8 Oct 30, 2026
32 of 35 checks passed
@TomasPalsson
TomasPalsson deleted the feat/automatic-reading-0.8.8 branch October 30, 2026 16:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant