A local document assistant for standalone servers and disconnected networks. Administrators upload PDF or Word documents; users ask questions and receive answers with document references. Ollama runs both the language model and the embedding model on the server.
RAG means retrieval-augmented generation: the application finds relevant passages in your documents and supplies them to the model along with the question. The uploaded documents become a searchable knowledge base.
| Area | Working functionality |
|---|---|
| Document ingestion | PDF, DOCX and legacy DOC uploads; Word paragraphs, tables, headers and footers; PDF page text; local English/Hebrew OCR for scanned pages |
| Knowledge management | Versioned document replacement, removal from search, and administrator-entered key/value knowledge facts |
| Questions and answers | Local semantic retrieval, recent conversation context, English/Hebrew answers, document/version/page citations, and insufficient-evidence responses |
| Conversations | Saved messages and citations, earlier conversation selection, pending-job recovery after a browser refresh, and account ownership checks |
| Administration | Create/list administrator and technician accounts; upload/remove documents; view processing jobs and failures |
| Access and operations | Local HTTPS, hashed passwords, server-side authentication, role checks, rate limits, dependency health checks, and retained job retries |
| Offline delivery | Bundled Docker images and Ollama models, Windows/Linux launchers, checksum verification, fresh installation credentials, and coordinated backup/restore tools |
The browser interface is in Hebrew with right-to-left layout. Answers follow the language of the question. Both roles use the shared knowledge base; conversation access is scoped to the signed-in account.
- An administrator uploads a document. The API checks its format and size, saves the source locally, and records an ingestion job in PostgreSQL. Redis signals that work is available.
- A background worker extracts text. Tesseract handles scanned pages; LibreOffice converts legacy Word files. These tools run locally.
- The worker splits the extracted text into overlapping chunks, retaining source and version details and PDF page numbers where available.
- Ollama runs
embeddinggemmato represent each chunk as a vector for semantic search. Chroma stores the text and vectors. - PostgreSQL publishes the indexed document generation as active. A replacement becomes searchable after indexing succeeds; a failed replacement leaves the previous version available.
For example, uploading Network_Guide_v2.docx after Network_Guide_v1.docx updates the same topic. A separate base filename represents a separate topic.
- The API checks authentication, conversation ownership and rate limits, then creates a high-priority chat job. The browser receives a job ID immediately.
- A chat worker retrieves relevant chunks from active document generations and adds bounded recent conversation history.
- Ollama runs
qwen3:4b-instructwith the question and that evidence. The prompt asks for references such as[S1]and an answer in the question's language. - The application checks that citation IDs refer to supplied evidence, then saves the answer, sources and job result in PostgreSQL. Missing evidence or invalid citation IDs produce an insufficient-evidence response.
- The browser polls the job and displays the saved answer. Reloading the page can recover the conversation and pending work.
Citations identify the document, version and PDF page when available. Check important claims against the original source: valid citation IDs do not establish that every generated sentence is accurate.
flowchart TD
Browser[Browser: React interface] --> Web[nginx: HTTPS and API proxy]
Web --> API[FastAPI: accounts, conversations and documents]
API --> SQL[(PostgreSQL: users, messages, jobs and document registry)]
API --> Redis[(Redis: worker wake-up signals)]
Redis --> Workers[Chat and ingestion workers]
Workers --> SQL
Workers --> Extract[Local extraction: PDF, Word, OCR and conversion]
Workers --> Ollama[Ollama: generation and embeddings]
Workers --> Chroma[(Chroma: document chunks and vectors)]
PostgreSQL is the durable record of jobs and active document versions. Redis helps wake workers; the database retains queued work. Separate chat and background workers give questions priority over ingestion. Cleanup and optional website-sync workers handle maintenance.
The backend uses Python/FastAPI, SQLAlchemy and LangChain integrations; the frontend uses React/Vite and nginx. Docker Compose starts the services, initializes the database and waits for dependencies and required models.
The package targets Linux amd64 containers and requires Docker Engine plus Compose 2.24.4 or newer already installed. Windows Desktop uses Docker Desktop in Linux containers mode. For Windows Server, run the Linux package inside a Linux VM with Docker Engine; see host requirements.
The transfer archive contains application source, Docker images and both models. It excludes the preparation machine's documents, accounts and history. Host Python, Node and Ollama installations are unnecessary.
tar -xf local-rag-standalone-2.2.0.tar
cd local-rag-standalone-2.2.0
Before the first launch, edit run-config.json to set server_names to the server's LAN hostname/IP and choose ports if the defaults are occupied.
| Platform | Start | Status | Stop |
|---|---|---|---|
| Windows Desktop | RUN.cmd |
RUN.cmd status |
RUN.cmd stop |
| Linux | bash RUN.sh |
bash RUN.sh status |
bash RUN.sh stop |
First launch verifies and imports the images/models, generates configuration and HTTPS certificates, starts the services, and creates an administrator with a random password. It starts with an empty knowledge base. The runner prints the address and login; the initial credentials are saved privately in runtime/admin-login.txt.
Trust the public certificate app/deploy/tls/ca.crt on browser clients, sign in over HTTPS, and upload your documents. Allow about 40 GB free disk for installation, plus space for documents and backups. Stopping preserves the data. Database content also lives in Docker named volumes; copying the application folder alone does not move it.
See installation, daily operation and backup/restore for the complete procedures.
A fresh Git checkout contains code, dependency locks and example configuration. Building images and downloading models happen on a connected preparation machine. Follow source installation before starting the application.
An already prepared checkout starts with:
bash server/scripts/start-local.shRuntime settings are defined in .env.example and loaded from .env. The standalone runner generates app/.env from its run configuration. Models must match the configured names and be present locally.
The standalone deployment applies the internal-network overlay. For source deployments, use the offline validation overlay to enforce blocked backend internet access. The base Compose file keeps internal services unpublished but does not itself enforce that isolation.
| Path | Purpose |
|---|---|
backend/app/ |
API, authentication, database models, retrieval, ingestion, jobs and workers |
frontend/src/ |
Chat interface, conversation recovery and administrator screens |
deploy/ |
Image pins, Compose overrides, proxy/TLS configuration and deployment/recovery guides |
server/scripts/ |
Release preparation, packaging, startup, certificate and backup/restore tools |
server/standalone/ |
Portable launchers, installation helper, run configuration and installer tests |
redis/ |
Redis runtime configuration |
tests/ |
Application regressions, browser acceptance and live offline/recovery checks |
docs/ |
Installation, operation, troubleshooting, testing and GitHub publishing instructions |
Application settings, permissions, version rules and database definitions are implemented in the Python code and active configuration. Dependency locks, tests and deployment scripts belong in the repository because they support reproducible setup and verification.
Publish the source, example configuration, lockfiles, tests and documentation. The following local installation files are excluded by .gitignore:
| Local files | Why they stay outside Git |
|---|---|
.env and runtime/ |
Generated credentials and installation state |
data/ except .gitkeep |
Uploaded documents, model stores and local account files |
deploy/tls/ and deploy/pki/ |
Installation certificates and private keys |
offline/ |
Large release archives, model/image exports, private backups and recovery copies |
| Dependency/build caches and database exports | Generated content that is prepared separately |
The standalone transfer archive is about 7.37 GB and is distributed separately from the source repository. Publishing instructions explain how to review the Git contents and push to your repository.
Release validation on the preparation machine included:
| Check | Result |
|---|---|
| Python application regressions | 60 tests passed, plus 5 version identity examples |
| Standalone preparation and launcher tests | 29 checks passed |
| Release integrity tools | 6 checks passed |
| Production browser acceptance | 10 check groups passed |
| Fresh offline installation and live document/chat acceptance | 19 checks passed; repeat launch preserved configuration and credentials |
GitHub Actions is configured to run application/tool tests, shell syntax checks and a frontend build without Ollama models or private data. Actual OCR, local model generation, recovery and capacity checks use the prepared release environment. See testing for repeatable commands; verify the intended server separately.
- The default deployment uses CPU execution and one generation at a time. GPU access requires additional host and Compose configuration; capacity depends on the selected models and hardware.
- OCR depends on scan quality. Encrypted, blank, damaged or unsupported documents can fail visibly during ingestion.
- Recent conversation context and retrieved evidence have bounded prompt budgets. A relevant document can still produce an incomplete answer.
- Default retention is seven days for chat messages and thirty days for finished jobs. Uploaded knowledge documents have no automatic expiry.
- Website synchronization and vision processing are implemented but disabled by default. Sync requires access to its configured site; vision requires an additional local model.
- Normal generation, embeddings and extraction require no cloud API. Telemetry and remote tracing are disabled in the supplied deployment configuration.