feat(retrieval): return a short document whole, and the top evidence page's neighbour

[hallelx2]

Vectorless now returns a short document whole, and fills open slots with the page beside the top evidence page before pages from elsewhere.

Why

Three of Vectorless's nine FinanceBench declines in the 2.5 Flash-Lite pilot were retrieval misses:

• Short filings. FOOTLOCKER_2022_8K came back as 2 sections. MGMRESORTS_2022Q4_EARNINGS came back as 5 small sections, without the financial table (gold page 14).
• Adjacent page. BOEING_2022_10K returned pages 7 and 22; the answer is on page 8.

Change

• Whole short documents. A document whose text fits in retrieval.judgewalk.whole_document_tokens comes back entire, whatever max_sections is. The default is 12,000 tokens; negative turns it off.
  • The judgewalk result carries WholeDocument plus DocumentPages (paged) or DocumentIDs (section tree).
  • Navigation's ranked units lead and the rest follow in document order, marked whole_document.
• Neighbour pages. The page after, then before, the top evidence page fills open slots before runner-ups. These pages are not evidence themselves, and keep their score when navigation read them. Each is marked neighbour_of.
• All paths. /v1/query, both /v1/answer paths and the standalone server take both through AssembleUnits. unit_stats counts neighbours and whole documents.

Verification

• Lorx CI run 39 passed (fmt, vet, build, lint, test).
• New tests:
  • a short paged document and a short section-tree document come back whole, past the cap, ranked units first;
  • the neighbour fills before runner-ups and is counted;
  • neighbours are next-then-previous and never evidence;
  • the threshold's default and its off switch.

Not yet measured

The issue's done-when is a re-measure of the 50 pilot questions on vlb-engine. That waits until the running pilot finishes, so the pilot keeps one engine build; I'll add the numbers here before merging.

Closes HAL-3246

Summary by Sourcery

Improve judgewalk retrieval by returning short documents whole and prioritizing pages adjacent to top evidence.

New Features:

• Return documents below the configurable size threshold in full, with navigation-ranked content first and remaining content in document order.
• Prioritize the page after and then before the top evidence page when filling retrieval slots, while marking these pages as neighbours rather than evidence.

Bug Fixes:

• Improve retrieval of answers spanning short filings or adjacent pages.

Enhancements:

• Propagate whole-document and neighbour metadata through query and answer assembly, including unit statistics and response fields.
• Expose configurable judgewalk whole-document thresholds with a default and an explicit disable option.

Tests:

• Add coverage for whole paged and section-tree documents, neighbour prioritization and ordering, evidence exclusion, statistics, and threshold behavior.

[coderabbitai]

Important

Draft PR not reviewed

Draft PRs are not automatically reviewed by default.

• Trigger a manual review

To automatically review draft PRs, update your CodeRabbit configuration:

reviews:
  auto_review:
    drafts: true

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[sourcery-ai]

Reviewer's Guide

JudgeWalk now returns short documents in full and fills remaining retrieval slots with the next, then previous, page beside the top evidence; configuration, assembly, query/answer paths, metadata, and unit statistics are updated consistently.

Sequence diagram for JudgeWalk neighbour-page retrieval

sequenceDiagram
    participant QueryHandler
    participant JudgeWalkStrategy
    participant PageStore
    participant AssembleUnits

    QueryHandler->>JudgeWalkStrategy: Select(ctx, tree, query)
    JudgeWalkStrategy->>PageStore: Load document pages
    PageStore-->>JudgeWalkStrategy: pages
    JudgeWalkStrategy->>JudgeWalkStrategy: neighbourPages(nav, pages, leafTitle)
    JudgeWalkStrategy-->>QueryHandler: Result with EvidencePages and NeighbourPages
    QueryHandler->>AssembleUnits: AssembleUnits(ctx, tree, result, k, load)
    AssembleUnits->>AssembleUnits: AddPage(neighbour, true)
    AssembleUnits-->>QueryHandler: Units with neighbour_of and unit_stats.neighbours

Loading

Flow diagram for whole short-document assembly

flowchart LR
    A[JudgeWalk selects document] --> B{Document within whole_document_tokens?}
    B -- No --> C[Assemble evidence and capped fallback units]
    B -- Yes --> D[Result sets WholeDocument]
    D --> E[Add ranked evidence and runner-ups]
    E --> F[Add remaining pages or sections in document order]
    F --> G[Return all units without cap]
    G --> H[Mark appended units whole_document]

Loading

File-Level Changes

Change	Details	Files
Propagate configurable short-document handling through JudgeWalk and expose whole-document results.	Added the whole_document_tokens setting with a 12,000-token default and negative disable switch. Detect short paged and section-tree documents and return their complete contents alongside ranked navigation results. Passed the JudgeWalk configuration into all server strategy construction paths.	cmd/server/main.go pkg/config/config.go pkg/retrieval/judgewalk.go pkg/retrieval/strategy.go
Prioritize adjacent pages when assembling bounded retrieval results.	Select the next page and then previous page beside the top evidence page, excluding evidence pages themselves. Preserve navigation scores and mark inserted pages with their evidence-page relationship. Count neighbour fills separately from runner-ups in unit statistics.	pkg/retrieval/judgewalk.go pkg/retrieval/strategy.go pkg/retrieval/units.go pkg/retrieval/units_test.go
Centralize whole-document and neighbour assembly across query and answer flows.	Updated paged and section-based results to carry whole-document and neighbour metadata into AssembleUnits. Return ranked units first, followed by remaining document content in document order without the normal section cap. Expose whole_document and neighbour_of markers in query output and propagate data through both answer handlers.	pkg/retrieval/units.go pkg/retrieval/units_test.go internal/handler/query.go internal/handler/answer.go internal/handler/answer_pages.go

---

Tips and commands

Interacting with Sourcery

• Trigger a new review: Comment @sourcery-ai review on the pull request.
• Continue discussions: Reply directly to Sourcery's review comments.
• Generate a GitHub issue from a review comment: Ask Sourcery to create an
  issue from a review comment by replying to it. You can also reply to a
  review comment with @sourcery-ai issue to create an issue from it.
• Generate a pull request title: Write @sourcery-ai anywhere in the pull
  request title to generate a title at any time. You can also comment
  @sourcery-ai title on the pull request to (re-)generate the title at any time.
• Generate a pull request summary: Write @sourcery-ai summary anywhere in
  the pull request body to generate a PR summary at any time exactly where you
  want it. You can also comment @sourcery-ai summary on the pull request to
  (re-)generate the summary at any time.
• Generate reviewer's guide: Comment @sourcery-ai guide on the pull
  request to (re-)generate the reviewer's guide at any time.
• Resolve all Sourcery comments: Comment @sourcery-ai resolve on the
  pull request to resolve all Sourcery comments. Useful if you've already
  addressed all the comments and don't want to see them anymore.
• Dismiss all Sourcery reviews: Comment @sourcery-ai dismiss on the pull
  request to dismiss all existing Sourcery reviews. Especially useful if you
  want to start fresh with a new review - don't forget to comment
  @sourcery-ai review to trigger a new review!

Customizing Your Experience

Access your dashboard to:

• Enable or disable review features such as the Sourcery-generated pull request
  summary, the reviewer's guide, and others.
• Change the review language.
• Add, remove or edit custom review instructions.
• Adjust other review settings.

Getting Help

• Contact our support team for questions or feedback.
• Visit our documentation for detailed guides and information.
• Keep in touch with the Sourcery team by following us on X/Twitter, LinkedIn or GitHub.