Repository navigation
One retrieval engine; every local engine picks a GPU that works and recovers from faults - #201
Closed
alichherawalla wants to merge 7 commits into
Closed
alichherawalla wants to merge 7 commits into
alichherawalla wants to merge 7 commits into
Conversation
Desktop now runs every short-record search on @offgrid/rag/records (src/main/records): screen moments, chats, meetings, to-dos, entities, messages and voice, through one keyword + vector pipeline with time windows, entity recall, reranking and dated, cited source blocks. - Search: the palette and Search screen keep their contract on the new engine; typed words match as prefixes, time words boost (or filter when alone), repeats collapse. - Index: one FTS tokenizer with a trigram twin for Chinese/Japanese/Thai; indexes rebuild in the background in slices; a non-empty source never gets live triggers on an empty index, and the small indexes self-check at startup (fixes SQLITE_CORRUPT_VTAB on meetings). - Vectors: multilingual-e5-small in record_vectors (SQLCipher) with a lazy int8 index; LanceDB is removed. Background indexing waits while a search runs and embeds 4 records a batch, so a search no longer queues behind it (2.7-8.5 s -> 0.2-0.7 s). - macOS embeddings never use Core ML (it ended the app mid-batch) and use WebGPU only when chosen in Settings: a WebGPU GPU fault wedged the chat model and froze the Mac. - Chat: every system prompt ends with today's date and earlier turns carry their day, so old "today"s are read as their own day. search_memory returns dated [S#] sources; [S#] tags are clickable again (react-markdown was blanking cite: links) and a cited chat opens in Chat. - Retired the legacy memories, facts and master memory paths, the chat-export parsers, OCR reprocessing and their IPC; rows stay in the database. - E2E: unified-retrieval.spec (cited dated memory, today's date, prefix/Hindi/Japanese/"today" search) and app068 waits for the document engine's ready row.
- Unused exports become module-private; dead symbols and files are removed (vision coordinate grid, the retired search-ranking, FTS-match and vector-predicate tests). No behaviour change. - Unused packages removed: @dnd-kit/sortable, apache-arrow, class-variance-authority, cmdk. - Database tests follow the legacy-memory purge; Pro rolls to its dead-code removal.
…and says what it runs on
- One backend policy (@offgrid/models) for chat, grounding, decision, image, transcription,
voice and search; desktop keeps the ports (engine scan, device probe, driver reader).
Settings offers only choices this machine can run, each disabled one with its reason.
- "CPU" means CPU only: llama --device none, whisper -ng, sd --backend cpu, also when only a
GPU build ships (macOS). No default -ngl 99: llama.cpp fits layers to VRAM itself.
- A GPU fault ("Compute error", CUDA error, device lost) restarts the same model, then runs it
on the CPU with one notice; the model never changes. Grounding and decision share one
supervised runtime with the same ladder and crash budget. Engines are checked on wake.
- Voice and search run in Electron utility processes: a driver crash ends a helper, not the app.
The CUDA library path reaches the helpers at spawn, so Linux no longer relaunches itself; on
Windows the pack's CUDA addon loads only after a CUDA run passed in a helper.
- NVIDIA driver floors gate CUDA and the pack. The pack install checks disk space, never ends in
HTTP 416, has no extract timeout, uses System32 tar, smoke-tests the GPU, needs no restart,
and installs under Local AppData on Windows.
- Image jobs free chat, grounding and decision; Qwen-Image keeps CPU offload unless the GPU is
known to hold it. Settings shows device, layers on GPU, fallback reason or last error, with
"Try the GPU again". Kev, Parakeet and ExecuTorch-only voices are not offered without their
runtime.
- Packaging: app-local VC++ runtime on Windows, libvulkan in the deb and beside Linux engines,
onnx-cuda out of installers, Windows Vulkan whisper build, macOS sd rebuilt for macOS 13 and a
minimum-OS gate over every engine.
- E2E: gpu-fault-recovery.spec (restart, CPU fallback notice and answer, Settings row, retry).
|
Important Review skippedToo many files! This PR contains 288 files, which is 188 over the limit of 100. To get a review, reduce the PR to 100 files or fewer by splitting it into smaller PRs or changing its base branch. Upgrade to a paid plan to raise the limit. This review couldn't start because sufficient usage credits or metered capacity aren't available. Add credits or update usage-based reviews in the billing tab, then retry. ⚙️ Run configuration
⛔ Files ignored due to path filters: 1
📒 Files selected for processing: 288
You can disable this status message by setting the
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…images fail loudly - The browser task returns the captured frame with each semantic observation, so the model sees what the accessibility snapshot describes. - A model request that requires an image refuses to run without a readable one, asking for a fresh screenshot instead of answering blind. - Task execution plans and their tests follow; e2e screenshots re-captured.
…etrieval # Conflicts: # e2e/app068-project-knowledge-isolation.spec.ts # e2e/screenshots/chat-composer-filled.png # e2e/screenshots/chat-screen.png # e2e/screenshots/meeting-transcription-picker.png # e2e/screenshots/memory-all-memory-selected.png # e2e/screenshots/memory-dropdown-open.png # e2e/screenshots/memory-no-memory-selected.png # e2e/screenshots/streaming-after-send.png # e2e/screenshots/streaming-placeholder.png # package-lock.json # package.json # pro # src/main/__tests__/search-ranking.test.ts # src/main/accessibility/decision-runtime.ts # src/main/chat-fallback.ts # src/main/database.ts # src/main/embeddings.ts # src/main/index.ts # src/main/ipc.ts # src/main/llm.ts # src/main/llm/settings-math.ts # src/main/mcp-server.ts # src/main/modality-queue/__tests__/jobs.test.ts # src/main/modality-queue/queue.ts # src/main/model-server.ts # src/main/models-manager.ts # src/main/native-helper-health.ts # src/main/runtime-manager.ts # src/main/search-ranking.ts # src/main/tools.ts # src/main/transcription/whisper-server-transcription.ts # src/main/tts-onnx.ts # src/main/tts.ts # src/main/vision/grounder-runtime.ts # src/renderer/src/components/MemoryChat/components/MessageMarkdown.tsx # src/renderer/src/components/SettingsPanel.tsx # src/renderer/src/components/SettingsSelect.tsx
…d vector store, and the retrieval engine and GPU recovery have tests - Import cycles (18): the NVIDIA driver facts live in nvidia-driver.ts, the backend facts cache in its own leaf, and the Windows ONNX CUDA check record in onnx-cuda-verified.ts. A pack install no longer reaches up into voice, search and best settings; setupIPC refreshes them when it lands. - Delete All also erases the retired LanceDB folder. It held a plaintext copy of captured text, and its removal at first launch does not wait to finish. - Tests of retired paths now drive what replaced them: memory through the chat tools loop and search_memory, user text with FTS syntax through universalSearch, the sessionSummary prompt, and captured moments instead of the memories table. Hand-written capture tables match Pro's schema. - New journeys: meaning search over record vectors (queue, drain, int8 index, kinds, time window, edits and deletes), each record source's time and short-script paths, the keyword and passage index rebuilds on an upgraded profile, wake health checks, the ONNX CUDA helper environment, helper process crash limits, and dedicated llama-server restarts with their budget. - Earlier turns carry their day and sibling-chat hits their conversation and time; the history and search tests compare what matters.
|
Contributor
Author
alichherawalla
added a commit
that referenced
this pull request
Oct 23, 2026
#201 made image jobs evict grounding and decision (they cannot share GPU memory); a clip needs more memory than a picture. Found in the merge-resolution review.
alichherawalla
added a commit
that referenced
this pull request
Oct 23, 2026
Only arm64 ships, so the arch suffix from #201 is not needed.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
One branch across shared, desktop (OGAD) and desktop-pro. Merge order: shared, desktop-pro, then OGAD.
What it does
One retrieval engine. Memory, search, meetings and to-dos run on
@offgrid/rag/records: one keyword + vector pipeline over screen moments, chats, meetings, to-dos, entities, messages and voice. Answers come with dated, clickable[S#]citations, and chat knows today's date. LanceDB is removed; vectors live in SQLCipher (multilingual-e5-small). The legacy memories/facts/master-memory paths are retired, and their rows stay in the database.Dead code removed in all three repos (unused exports, symbols, files; four unused packages).
GPU backends that work and say what they run on:
@offgrid/modelsfor chat, grounding, decision, image, transcription, voice and search. Settings offers only choices this machine can run, and says why when it cannot.--device none,-ng,--backend cpu). No default-ngl 99: llama.cpp fits layers to VRAM itself.onnx-cudakept out of installers, a Windows Vulkan whisper build, macOSsdbuilt for macOS 13, and a minimum-OS gate.Verified
e2e/unified-retrieval.specand the newe2e/gpu-fault-recovery.spec(same-model restart, CPU fallback notice and answer, Settings row, "Try the GPU again") pass on macOS.Not verified / known
--no-verify. The shared pre-push hook still fails Pro tests that predate this work:capture-projection-cleaner,crm/synthesize.npm run test:dbswaps it).