The Professional Offline-First Local LLM Benchmark & Playground Tool
The ultimate desktop application to evaluate, profile, and compare LLM execution speeds and reasoning scores on your own hardware.
Download v1.0.0 · Features · Quick Start · Architecture · Packaging · Testing
Get the official installer for your operating system:
| Platform | Download Link | Package Format | Architecture |
|---|---|---|---|
| 🍏 macOS (Apple Silicon) | Download for Mac (ARM64) | .dmg installer |
Apple Silicon (M1/M2/M3/M4) |
| 🍏 macOS (Intel) | Download for Mac (x64) | .dmg installer |
Intel Processors |
| 🪟 Windows | Download for Windows | .exe installer |
Intel x64 / ARM64 |
No more guessing whether a model will fit in your memory or crash your device.
- Unified Memory Analysis: Deep hardware inspection for Apple Silicon (Unified Memory limits) and Windows/Intel platforms (Dedicated GPU VRAM).
- Suitability Categorization: Flags models dynamically as Butter (comfortably fits), Struggle (CPU offloading, resource bottleneck), or Unrecommended (will swap or crash).
- Direct Library Integration: Browse, pull (with live layer-by-layer download progress metrics), and delete models directly from Ollama within the UI.
- Auto-Sync Registry: Newly downloaded models automatically populate selectors across the application in real-time.
Evaluating multiple local models concurrently thrashing your GPU, skewing your benchmark scores, or triggering Out-of-Memory (OOM) errors. LocalBench runs custom evaluation runs sequentially, model-by-model, while tasks within a single model can execute concurrently for optimized speed profiling.
Safely test local outputs against frontier baselines like GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro.
- Secure Key Storage: API credentials reside exclusively on the client-side
localStorageand are only injected during active requests. - OpenRouter Integration: Access thousands of open-source models hosted in the cloud.
Compare responses side-by-side (up to 3 models concurrently).
- Real-Time Streaming: Stream outputs from both local runners and cloud APIs.
- Live Performance Metrics: Tracks and displays Time to First Token (TTFT), Tokens Per Second (TPS), and total duration for each model's response.
Evaluates response quality using flexible, layout-insensitive scoring algorithms:
- Speed (TPS): Pure velocity measurement.
- Reasoning (Logic/Math): Deductive multi-turn verification.
- Syllogisms: Sequenced logical extraction (insensitized to markdown headers or numbering formatting).
- Coding: Identifies Python docstrings, asserts, syntax structures, and specific bug fixes.
- Translation: Cross-lingual comparison.
- Creative Writing: Verifies syllable distributions and structural counts (e.g. Haiku checks).
graph TD
A[React App Store / Zustand] -->|IPC Invocation| B[Electron Preload]
B -->|IPC Handlers| C[Electron Main Process]
C -->|Local API Queries| D[Local Providers: Ollama, Jan, LM Studio]
C -->|REST Requests + Local Keys| E[Cloud Providers: OpenAI, Anthropic, Gemini, OpenRouter]
C -->|Read/Write Results| F[(SQLite Database)]
- Frontend: React 18, Zustand (State Management), TailwindCSS, Recharts (Data Visualization), Lucide React.
- Backend (Electron): Main process IPC router, SQLite cache database for test historical data, Child process spawn wrappers (
ollama pullstreaming). - Bundler: Vite + TypeScript.
- Node.js (v18+ recommended)
- Ollama running locally (optional, but highly recommended for local models)
npm installThis will compile the Electron main process, start the Vite server, and spin up the Desktop window.
npm run devBuild target packages for production deployment:
npm run build:macnpm run build:winPackaged applications will be exported into the ./dist directory.
Ensure all scorers and pulling utilities are fully tested:
npm test