Skip to content

Repository files navigation

DocRAG — Fully Offline Document Intelligence Assistant

100% LOCAL • PRIVATE • CITATION-GROUNDED • PERMANENT DOCUMENT LIBRARY


DocRAG is a fully offline document intelligence and Retrieval-Augmented Generation (RAG) system designed to let users build a permanent personal library of documents and ask questions across that library.

It supports:

  • PDF
  • DOCX
  • TXT

Documents are parsed, intelligently chunked, embedded and permanently stored in LanceDB on the local computer.

The system then uses hybrid search — vector search + BM25 — to retrieve relevant passages before generating an answer through a locally running Qwen2.5-Instruct model.

Every answer is designed to be grounded in the ingested document text and accompanied by numbered citations showing the source and page.


#### ⭐ WHAT MAKES DocRAG DIFFERENT?

DocRAG is built around one simple idea:

Your documents become a permanent local knowledge library that you can continue expanding over time.

Instead of uploading a document every time you want to ask a question, DocRAG permanently stores the processed document data locally.

Drag Book / Document
        ↓
Parse
        ↓
Chunk
        ↓
Embed
        ↓
LanceDB
        ↓
Permanent Local Library
        ↓
Ask Questions Anytime
        ↓
Hybrid Retrieval
        ↓
Citation-Grounded Answer

Once a document has been ingested, it remains available after:

  • Closing DocRAG
  • Restarting the application
  • Rebooting Windows
  • Adding additional documents
  • Returning to the system later

Documents remain stored until the user deliberately deletes them.


#### 🔒 FULLY OFFLINE & LOCAL OPERATION

DocRAG is designed to operate entirely on the local Windows computer after the required dependencies, runtime files and models have been prepared.

                    INTERNET
                       X
                       |
                       X
                  DocRAG SYSTEM
                       |
        ┌──────────────┼──────────────┐
        ↓              ↓              ↓
     Qwen2.5         bge-m3        LanceDB
      Local          Local          Local
        ↓              ↓              ↓
        └──────────────┼──────────────┘
                       ↓
                  LOCAL ANSWER

Runtime communication takes place through localhost.

There is no requirement for:

  • Cloud AI
  • Cloud database
  • Remote RAG service
  • External document storage
  • Online inference

The only Internet requirement described by the project is the one-time pip install required to prepare the Python environment.

After the required components are installed and the models are present, the application is designed for localhost-only operation.


#### 🧠 CORE AI ARCHITECTURE

DocRAG uses two local AI model services.

1. Qwen2.5-Instruct

Used for:

  • Question answering
  • Document-grounded response generation
  • Professional language generation

Supported model tiers:

Qwen2.5 3B
Qwen2.5 7B
Qwen2.5 14B

The appropriate model can be selected automatically according to detected hardware and available model files.


2. bge-m3

Used as the local embedding model.

It converts document chunks into numerical representations that allow DocRAG to identify semantically relevant passages.

Document
   ↓
Text Chunk
   ↓
bge-m3
   ↓
Vector Embedding
   ↓
LanceDB

#### 🏗️ SYSTEM ARCHITECTURE

                         USER
                           │
                           ▼
                 ┌──────────────────┐
                 │ DrgFrontend.html │
                 │  HTML + CSS + JS │
                 └────────┬─────────┘
                          │
                     fetch() / JSON
                          │
                          ▼
                 ┌──────────────────┐
                 │  DrgServer.py   │
                 │    :8000        │
                 └────────┬─────────┘
                          │
            ┌─────────────┼─────────────┐
            │             │             │
            ▼             ▼             ▼
       DrgIngest.py  DrgQuery.py   DrgCore.py
            │             │
            │             │
            ▼             ▼
        Documents     Hybrid Search
                          │
                 ┌────────┴────────┐
                 │                 │
                 ▼                 ▼
              Vector              BM25
              Search             Search
                 │                 │
                 └────────┬────────┘
                          ▼
                    Retrieved Text
                          │
                          ▼
                    Qwen2.5 Chat
                      :8080
                          │
                          ▼
                 Citation-Grounded
                       Answer
                          │
                          ▼
                    Browser UI

#### 📡 LOCAL SERVICE ARCHITECTURE

DocRAG uses separate local llama.cpp server instances.

DrgServer.py :8000
       │
       ├──────────────► llama-server
       │                 bge-m3
       │                 :8090
       │
       └──────────────► llama-server
                         Qwen2.5
                         :8080

The embedding model and chat model therefore operate as separate local services.


#### 📁 PROJECT STRUCTURE

DocRAG\
│
├── DrgCore.py
├── DrgHardwareCheck.py
├── DrgIngest.py
├── DrgQuery.py
├── DrgServer.py
├── DrgFrontend.html
├── DrgRun.bat
│
├── engine\
│   └── llama-server.exe
│
├── models\
│   ├── Qwen2.5-*.gguf
│   └── bge-m3*.gguf
│
├── vectorstore\
│   └── LanceDB data
│
└── SQLite database
    └── Query history

#### ⚙️ BACKEND COMPONENTS

DrgCore.py

Central backend configuration and infrastructure layer.

Handles:

  • Configuration
  • LanceDB access
  • Local llama-server clients
  • Connection information
  • Core shared functionality

DrgHardwareCheck.py

Automatically detects available computer hardware.

It checks:

  • RAM
  • GPU availability
  • Available model files

It then uses fuzzy filename matching to identify available:

Qwen2.5 3B
Qwen2.5 7B
Qwen2.5 14B

This allows DocRAG to adapt its model selection to the computer.


DrgIngest.py

Responsible for document ingestion.

Supported formats:

PDF
DOCX
TXT

The workflow is:

Document
   ↓
Parser
   ↓
Text Extraction
   ↓
Chunking
   ↓
bge-m3 Embedding
   ↓
LanceDB

The resulting vector data is stored permanently on disk.


DrgQuery.py

Responsible for document question answering.

It performs:

User Question
      ↓
Hybrid Retrieval
      ↓
Vector Search + BM25
      ↓
Relevant Document Chunks
      ↓
Citation-Grounded Prompt
      ↓
Qwen2.5
      ↓
Answer + Citations
      ↓
SQLite Query Log

DrgServer.py

FastAPI application that connects the frontend and backend components.

It:

  • Provides HTTP endpoints
  • Handles document uploads
  • Connects ingestion and query functions
  • Serves the frontend
  • Communicates with the local llama-server instances

Server:

http://127.0.0.1:8000

#### 💻 FRONTEND

DocRAG uses a single self-contained frontend:

DrgFrontend.html

It contains:

  • HTML
  • CSS
  • JavaScript

No frontend framework is required.


#### 👑 ROYAL LUXURY INTERFACE

The interface follows a Royal Luxury visual theme.

Design characteristics include:

Midnight Indigo
+
Gold
+
Burgundy
+
Serif Typography

The interface is designed to provide a professional document-library experience rather than a generic chat interface.


#### 📚 DOCUMENT BOOKSHELF

The frontend includes a visual document shelf.

Users can drag and drop books or documents into the interface.

DRAG DOCUMENT
      ↓
   UPLOAD
      ↓
   INGEST
      ↓
 PERMANENT
  STORAGE
      ↓
 DOCUMENT
  BOOKSHELF

The bookshelf represents the user's local document library.


#### ♾️ PERMANENT DOCUMENT MEMORY

One of DocRAG's major features is persistent document storage.

When a document is ingested, its processed information is stored in LanceDB on disk.

Therefore:

Today
  ↓
Add Book 1
  ↓
Add Book 2
  ↓
Add Book 3
  ↓
...
  ↓
Tomorrow
  ↓
Add Book 4
  ↓
Next Month
  ↓
Add Book 5
  ↓
Next Year
  ↓
All Previously Stored Books

There is no automatic expiration described in the system.

Documents remain available until they are deliberately deleted.


#### 📖 MULTI-BOOK QUESTION ANSWERING

If ten books are stored in DocRAG, a question searches across the entire document library.

For example:

Book 1 ─┐
Book 2 ─┤
Book 3 ─┤
Book 4 ─┤
Book 5 ─┤
Book 6 ─┤
Book 7 ─┼──► Hybrid Search ──► Answer
Book 8 ─┤
Book 9 ─┤
Book10 ─┘

A question may retrieve relevant passages from:

  • One book
  • Several books
  • Multiple chapters
  • Multiple sources

depending on which stored chunks are relevant to the question.

The system therefore functions as a growing personal document knowledge base.


#### 🔍 HYBRID SEARCH

DocRAG does not rely exclusively on semantic vector search.

It combines:

VECTOR SEARCH
      +
    BM25
      ↓
HYBRID RETRIEVAL

This allows the system to consider both:

  • Semantic similarity
  • Keyword/text relevance

The retrieved material is then supplied to the local Qwen2.5 model for answer generation.


#### 📑 CITATION-GROUNDED ANSWERS

DocRAG is designed to answer questions using the text retrieved from the user's stored documents.

The intended workflow is:

Question
   ↓
Search Library
   ↓
Retrieve Relevant Chunks
   ↓
Identify Source
   ↓
Identify Page
   ↓
Generate Answer
   ↓
Numbered Citations

Citations contain information identifying the relevant source and page.

This makes it easier to trace an answer back to the original document.


#### 🛡️ CITATION-FAITHFULNESS CHECK

DocRAG includes a citation-faithfulness checking mechanism.

Its purpose is to flag citations that are not sufficiently grounded in the retrieved material.

Generated Answer
       ↓
Citations
       ↓
Faithfulness Check
       ↓
Grounded?
   ↙       ↘
 YES       FLAG

This adds an additional verification layer between document retrieval and the final answer.


#### 📥 DRAG & DROP INGESTION

Documents can be added directly through the frontend.

Drag PDF / DOCX / TXT
          ↓
       Upload
          ↓
      Duplicate Check
          ↓
        Parse
          ↓
        Chunk
          ↓
       Embed
          ↓
       LanceDB

This removes the need to manually configure each document.


#### ♻️ DUPLICATE DETECTION

DocRAG includes duplicate-file detection.

If the same document is dragged into the application again, the system checks it rather than unnecessarily creating another duplicate entry.

This helps keep the document library clean.


#### 🗑️ DELETE-A-BOOK

Documents can also be removed from the library.

The bookshelf therefore provides both:

ADD DOCUMENT
      +
DELETE DOCUMENT

giving the user control over their local knowledge base.


#### 🕘 QUERY HISTORY

DocRAG logs queries using SQLite.

The architecture therefore includes:

User Question
      ↓
Answer
      ↓
SQLite
      ↓
Query History

This allows previous queries to be retained locally.


#### 🧠 AUTOMATIC HARDWARE DETECTION

DocRAG checks the computer before launching the AI system.

Computer
   ↓
Hardware Check
   ↓
RAM Detection
   ↓
GPU Detection
   ↓
Model Detection
   ↓
Select Appropriate Tier

The system can automatically choose between Qwen2.5:

3B
7B
14B

according to the detected environment and available model files.

The hardware tier determines the model used and therefore influences performance.


#### 🚀 ONE-CLICK LAUNCH

DocRAG is designed around a simple Windows launcher.

DrgRun.bat
     ↓
Check Folders
     ↓
Check Dependencies
     ↓
Hardware Detection
     ↓
Select Models
     ↓
Start llama-server
     ↓
Start FastAPI
     ↓
Open Browser
     ↓
DocRAG Ready

The goal is to make starting the complete local RAG system a one-click operation.


#### ▶️ HOW TO USE DocRAG

STEP 1 — Prepare llama-server

Place:

llama-server.exe

inside:

engine\

STEP 2 — Add the models

Place the required GGUF models inside:

models\

For example:

models\
├── Qwen2.5-7B-Instruct-*.gguf
└── bge-m3-q8_0.gguf

The exact Qwen2.5 model tier can vary according to the available hardware and model files.


STEP 3 — Start DocRAG

Double-click:

DrgRun.bat

STEP 4 — Wait for Ready

Allow the hardware check, model startup and FastAPI server to initialize.

When the interface reports:

Ready

DocRAG is ready.


STEP 5 — Add Documents

Drag and drop:

PDF
DOCX
TXT

into the document bookshelf.


STEP 6 — Ask Questions

Ask questions about the documents in your library.

DocRAG searches the stored document collection and generates a citation-grounded answer.


#### 🔄 COMPLETE DOCUMENT WORKFLOW

                  DOCUMENT
                      │
                      ▼
               Drag & Drop
                      │
                      ▼
                DrgIngest.py
                      │
                      ▼
                Text Parser
                      │
                      ▼
                   Chunk
                      │
                      ▼
                  bge-m3
                      │
                      ▼
                 Embedding
                      │
                      ▼
                  LanceDB
                      │
              PERMANENT STORAGE
                      │
                      ▼
                 User Query
                      │
                      ▼
                DrgQuery.py
                      │
             ┌────────┴────────┐
             ▼                 ▼
         Vector Search        BM25
             │                 │
             └────────┬────────┘
                      ▼
                Relevant Text
                      │
                      ▼
                  Qwen2.5
                      │
                      ▼
             Grounded Answer
                      │
                      ▼
             Source + Page
                Citations

#### 📊 SPECIFICATIONS

Component Technology
Backend Python
API Framework FastAPI
Frontend HTML5 + CSS3 + JavaScript
LLM Qwen2.5-Instruct GGUF
Model Tiers 3B / 7B / 14B
AI Runtime llama.cpp
Embeddings bge-m3 GGUF
Vector Database LanceDB
Query Logging SQLite
Search Vector + BM25
Input Formats PDF / DOCX / TXT
Server DrgServer.py
Frontend DrgFrontend.html
Launcher Windows BAT
Backend Port 8000
Chat Model Port 8080
Embedding Model Port 8090
Runtime Architecture Localhost

#### 🚫 WHAT DocRAG DOES NOT REQUIRE AT RUNTIME

DocRAG is designed around local execution.

It does not require:

❌ Cloud AI
❌ Cloud RAG
❌ Remote Vector Database
❌ Online Document Storage
❌ API Subscription
❌ External AI Service
❌ Internet Connection at Runtime

The project does require the one-time preparation of the Python environment through pip install, as specified in the project architecture.


#### 🔐 PRIVACY

DocRAG is designed for privacy-sensitive document work.

The intended data path is:

USER DOCUMENT
      ↓
LOCAL COMPUTER
      ↓
LOCAL PARSER
      ↓
LOCAL EMBEDDING MODEL
      ↓
LOCAL LANCEDB
      ↓
LOCAL QWEN2.5
      ↓
LOCAL ANSWER

Documents and queries remain within the local architecture during runtime.

No cloud document repository is required.


#### 📚 BUILD YOUR OWN PERSONAL LIBRARY

DocRAG can be used as a growing personal document repository.

For example:

Day 1
├── Book A
├── Book B
└── Book C

Day 30
├── Book A
├── Book B
├── Book C
├── Book D
├── Book E
└── Book F

Day 365
├── Book A
├── Book B
├── Book C
├── Book D
├── Book E
├── Book F
├── Book G
├── Book H
└── ...more documents

The documents remain available as long as their local LanceDB data remains intact and the user does not delete them.

The practical limit is therefore primarily determined by available disk storage and system performance, rather than a fixed book-count limit specified in the project description.


#### ⚠️ IMPORTANT RAG LIMITATIONS

DocRAG is designed to provide answers grounded in ingested documents, but retrieval-based AI is not infallible.

Important considerations include:

  • Retrieval may not always select the perfect passage.
  • Very similar or ambiguous documents may occasionally produce confusing results.
  • Large libraries containing unrelated subjects may increase retrieval complexity.
  • The system described here does not currently include a dedicated "search only this book" filter.
  • The project has been verified at small scale rather than being presented as stress-tested for hundreds or thousands of books over years.

Therefore, citation-grounded output should still be checked against the original document when accuracy is important.


#### 🔮 POSSIBLE FUTURE EXPANSION

The following features are identified as possible future improvements and are not currently built:

LAN Access

Allow access from:

Phone
Tablet
Other LAN Computer

The existing API architecture could be adapted for LAN use.


AMD / Intel GPU Detection

Current hardware detection is described as:

NVIDIA-focused

Future versions could expand automatic GPU detection to AMD and Intel hardware.


Stronger Reranking

A stronger reranking stage such as FlashRank could potentially be added after the existing retrieval pipeline.

Vector + BM25
      ↓
Initial Results
      ↓
Reranker
      ↓
Best Results
      ↓
Qwen2.5

Optional LoRA

A future LoRA layer could potentially be used to customize:

  • Tone
  • Formatting
  • Response style

after the core RAG workflow is established.


Multi-Book Cross-Referencing

A future GraphRAG-style architecture could enable deeper cross-document relationships.

For example:

Book A
   │
   ├──── Topic X
   │
   └──── Person Y
          │
          ▼
Book B ─── Topic X
          │
          ▼
Book C ─── Person Y

This could be particularly useful for research across multiple legal, academic or professional volumes.


#### 🎯 INTENDED USE

DocRAG is designed for:

  • Book research
  • Document research
  • Academic material
  • Legal documents
  • Professional reference libraries
  • Manuals
  • Reports
  • Technical documentation
  • Personal knowledge bases
  • Multi-document question answering
  • Citation-grounded research

#### 🏆 DocRAG HIGHLIGHTS

✓ Fully local RAG architecture
✓ Private document processing
✓ PDF support
✓ DOCX support
✓ TXT support
✓ Drag-and-drop ingestion
✓ Permanent LanceDB storage
✓ Persistent document library
✓ Duplicate-file detection
✓ Delete-a-book support
✓ Multi-book searching
✓ Vector search
✓ BM25 search
✓ Hybrid retrieval
✓ Qwen2.5 local inference
✓ bge-m3 local embeddings
✓ Citation-grounded answers
✓ Source + page citations
✓ Citation-faithfulness checking
✓ SQLite query history
✓ Automatic hardware detection
✓ Qwen 3B / 7B / 14B selection
✓ CPU/GPU-aware operation
✓ FastAPI backend
✓ Self-contained HTML frontend
✓ Local llama.cpp servers
✓ One-click Windows launcher
✓ No cloud RAG dependency
✓ Local document knowledge base

#### 🧩 DESIGN PHILOSOPHY

DocRAG is built around five principles:

PRIVACY

Keep documents and queries local.

PERSISTENCE

Build a document library that remains available until deliberately deleted.

GROUNDING

Answer questions from retrieved document content rather than relying solely on general model knowledge.

TRACEABILITY

Provide source and page citations so answers can be checked against the original material.

SIMPLICITY

Drag in a document, wait for ingestion, ask questions.


#### 📜 PROJECT

Project: DocRAG Version: Current Development Build Platform: Windows Architecture: Local / Offline / RAG Backend: Python + FastAPI AI Runtime: llama.cpp LLM: Qwen2.5-Instruct GGUF Embeddings: bge-m3 GGUF Vector Database: LanceDB Query Database: SQLite Frontend: HTML5 + CSS3 + JavaScript API Port: 8000 Chat Model Port: 8080 Embedding Port: 8090


#### 👤 BY ARYAN

DocRAG — Your Books. Your Knowledge Base. Your Answers.

A local document intelligence system designed to transform a growing collection of PDF, DOCX and TXT files into a searchable, citation-grounded personal knowledge library.


About

DocRAG is a fully offline, private document intelligence assistant that lets you permanently store and query PDF, DOCX and TXT files. Powered by Qwen2.5 and bge-m3 through llama.cpp, it searches your entire personal library and provides citation-grounded answers with source and page references.

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages