A single-binary, lightweight, production-grade Rust LLM routing gateway and dashboard with health-aware failover, rate-limit management, multi-key rotation, Claude Code compatibility, and portable zero-wear execution.
Vibe Router provides unified OpenAI (/v1/chat/completions) and Anthropic (/v1/messages) compatibility and routes requests across any combination of local and cloud providers with data-driven fallback chains (combos), automatic retries, exponential backoff, micro-dollar cost tracking, and an embedded Web UI.
- Single Lightweight Binary: Compiled to a ~8.5–9.5 MB standalone executable (≈8.5 MB on x86_64 Linux, ≈9.5 MB on Windows MSVC). No Node.js runtime, no Python, no external database dependencies.
- Tiny Memory Footprint: Reported ~9.2 MB RSS under live traffic plus the dashboard, versus ~265 MB for Node/Next.js gateways. On 1–2 GiB VPS instances, tune
cache.max_memory_mbdown (e.g.16or32) to bound the LRU cache; thevpsruntime profile already caps it at 4 MB. - Hardware-Aware Adaptive Runtime:
- Disk-First VPS Mode: Minimizes RSS footprint (< 30 MB) and restricts cache to 4 MB for small $5/mo VPS instances.
- RAM-First Portable Mode: Hydrates SQLite into host memory (
:memory:) with periodic crash-safe atomic flush back to disk, providing zero flash wear on USB drives. - Drive-Letter Independence: Normalizes paths with
{{PORTABLE_ROOT}}so changing USB drive letters on Windows never breaks the configuration.
- Universal Client Compatibility:
- OpenAI Compatible: Direct drop-in for OpenAI SDK, Cursor, Cline, OpenCode, and local agents at
http://localhost:20128/v1. - Claude Code (
/v1/messages): Native Anthropic protocol adapter supporting both JSON completions and real-time SSE streaming. Runvibe-router env claude-codeto connect Claude CLI in seconds.
- OpenAI Compatible: Direct drop-in for OpenAI SDK, Cursor, Cline, OpenCode, and local agents at
- Native
Vibe-LinkSynergy Protocol (with Vibe Assembly):- Telemetry response headers:
X-Vibe-Router-Cost-Micros,X-Vibe-Router-Latency-Ms,X-Vibe-Router-Cache, andX-Vibe-Router-Provider. - Inbound role tracking (
X-Vibe-Role: boss | code | debug | judge | chat | hermes) with per-role token & spend accounting. - Judge Retry Bypass (
X-Vibe-Retry: 1): Bypasses the cache and rotates to the next healthy candidate in the chain so code rejected by evaluation judges is retried on an alternative model. - Zero-401 Local Handshake: Synchronizes
.vibe_router_tokenfor seamless local agent authentication.
- Telemetry response headers:
- Multi-Key Pool Rotation, 429 Backoff & Auth-Fault Retirement:
- Supports
RoundRobin,LeastLoaded, andWeightedselection strategies across multiple API keys per provider. - Upstream 429 rate-limit responses trigger automatic exponential backoff cooldowns and instantly retry the next available healthy key transparently.
- A 401/403 on a pooled key is treated as a key fault, not a request fault: that key is retired for a long auth cooldown (
auth.auth_cooldown_ms, default 30 minutes, cleared on successful use or config reload) and the request rotates to the next healthy key — on both non-streaming and streaming paths. Only when every key is exhausted does the last 401/403 surface, unchanged and without a candidate-level cooldown wait. - Request-scoped errors (400/404/422 and validation failures) fail fast: they neither rotate keys nor retry.
- Supports
- High-Capacity In-Memory Prompt & Stream Cache:
- Bounded LRU cache (256 MB default; profile-tuned) hashing the full response-shaping surface —
model,messages,tools,tool_choice,temperature,top_p,max_tokens,stop,seed,n, penalties,response_format,logit_bias— via SHA-256 with sub-millisecond cache hits. - Opt-in stream caching cleanly buffers and replays completed SSE streams; cache hits are recorded as zero-cost usage rows.
- Bounded LRU cache (256 MB default; profile-tuned) hashing the full response-shaping surface —
- Virtual Model Aliasing:
- Map arbitrary model names (e.g.,
gpt-4o,claude-3-7-sonnet) to underlying combos or provider targets via REST, CLI, or Web UI.
- Map arbitrary model names (e.g.,
- Automatic
Vibe LocalNode Discovery:- Discovers local worker nodes on this PC and across the LAN via a UDP probe (
VIBE_DISCOVER_V1on port 20131, loopback + broadcast) with an HTTP fallback (GET http://127.0.0.1:20129/v1/models) for hosts that block UDP. - Responding nodes are registered as passthrough OpenAI providers (
vibe-localfor the canonical loopback node,vibe-local-<host>for LAN workers) and removed automatically when they stop responding. - Controlled by
server.auto_discover_local(defaulttrue). SetVIBE_LOCAL_TOKENin the environment to authenticate against secured nodes; the token is referenced as anenv:secret and resolved at request time, never stored literally.
- Discovers local worker nodes on this PC and across the LAN via a UDP probe (
- Remote Catalog Sync & Auto-Combo (
daily-free):- Point
server.remote_catalog_urlat aVibe_RadarJSON feed; the router refreshes it everyserver.remote_catalog_interval_secs(default 900s, 30s floor) on a bounded 15s / 2 MB budget. - Advertised models are merged onto existing providers without touching their configured endpoints or credentials; new catalog-only providers are SSRF-guarded (public
http(s)hosts only) and registered anonymously. - The feed's
daily_free_chainis filtered to resolvableprovider/modelentries and upserted as a runtime combo nameddaily-free, with afreealias registered when unset. Stale catalog providers and an empty chain are removed on the next pass.
- Point
- Circuit Breakers & Health Probes:
- Per-provider circuit breaker automatically trips to
OPENafter repeated 5xx errors or timeouts, transitions toHALF_OPENfor probing, and resets toCLOSEDon success.
- Per-provider circuit breaker automatically trips to
- Tagged Proxy Pools & Cloudflare Worker Relay:
- Support proxy tier tags (
cf-worker,datacenter,residential,direct). - Deterministic 64-bit FNV-1a sticky IP affinity ensures an API key consistently uses the same proxy to avoid anti-fraud bans.
- Built-in ready-to-deploy Cloudflare Worker reverse-proxy script (
vibe-router relay worker-script).
- Support proxy tier tags (
- Interactive Terminal UI & MCP Server:
vibe-router tui(ortop): Live terminal dashboard displaying RAM profile, spend, latencies, and circuit states.vibe-router mcp: Built-in stdio Model Context Protocol (MCP) server exposing tools to external AI assistants (vibe_router_status,vibe_router_costs,vibe_router_health,vibe_router_set_alias,vibe_router_add_proxy).
- Embedded Web Dashboard:
- Vanilla HTML/CSS/JavaScript interface embedded via
rust-embedwith industrial dark/yellow branding (#F5B800), live token counters (distinguishing reported vs. estimated), USD cost & p50/p95/p99 latency metrics, combo editor, and API playground. - Session-key authenticated: the dashboard stores the admin key in
sessionStorageand sends it asAuthorization: Beareron every/api/*call. All dynamic values are rendered with safe DOM APIs (noinnerHTMLinterpolation) under a strict Content-Security-Policy.
- Vanilla HTML/CSS/JavaScript interface embedded via
- Hardened Admin Boundary:
- Every
/api/*route requires the configured API key; state-mutating requests must carryContent-Type: application/jsonorX-Vibe-Admin: 1(CSRF guard). - Runtime-created providers cannot use
env:/file:secret references or shadow config-file provider IDs, and theirbase_url/proxy URLs are SSRF-guarded (loopback, link-local, and RFC1918 targets blocked unlessserver.allow_local_upstreams = true).
- Every
- Unified Storage:
- A single SQLite owner serves both the sync request path and the async admin API under one schema (
providers,combos,aliases,proxies,rates,usage), so every request is recorded exactly once and priced from one persisted rate table.
- A single SQLite owner serves both the sync request path and the async admin API under one schema (
Vibe Router is structured as a modular 11-crate Rust workspace with strictly downward, acyclic dependency flow:
vibe-router/
├── Cargo.toml # Workspace manifest
├── ARCHITECTURE.md # Architecture reference
├── vibe-router.toml.example # Documented sample configuration
├── src/main.rs # Minimal CLI driver entrypoint
└── crates/
├── vibe-router-core/ # Core traits, Model, Combo, Candidate, WireFormat, HardwareProfile
├── vibe-router-config/ # TOML parsing, validation, and error classification rule engine
├── vibe-router-health/ # Availability tracking: cooldowns, backoff levels, RPM limits, circuit breakers
├── vibe-router-routing/ # Fallback engine: candidate resolution, retry loop, failover driver
├── vibe-router-translate/ # Wire format translation (OpenAI, Anthropic /v1/messages, SSE decoders)
├── vibe-router-providers/ # HTTP transport, multi-key rotation pools, sticky proxies, CF relay
├── vibe-router-storage/ # Single SQLite owner (sync Store + async StorageBackend), WAL, portable hydration
├── vibe-router-cache/ # In-memory bounded LRU prompt & SSE stream cache with SHA-256 keys
├── vibe-router-ui/ # Embedded static Web UI assets (HTML/CSS/JS) via rust-embed
├── vibe-router-server/ # Axum server: /v1/*, /api/*, embedded dashboard, and Vibe-Link handlers
└── vibe-router-cli/ # Command-line interface, TUI, and stdio MCP server
Ensure Rust 1.85+ is installed:
cargo build --releaseThe resulting binary will be at target/release/vibe-router (or target/release/vibe-router.exe on Windows).
Copy the example configuration:
cp vibe-router.toml.example vibe-router.tomlEdit vibe-router.toml with your providers and keys:
[server]
host = "127.0.0.1"
port = 20128
api_keys = [] # Optional local auth keys
[[providers]]
id = "groq"
display_name = "Groq Cloud"
base_url = "https://api.groq.com/openai/v1/chat/completions"
auth = { scheme = "Bearer", secret = "env:GROQ_API_KEY" }
models = [
{ id = "llama-3.3-70b-versatile" }
]
[[providers]]
id = "ollama"
display_name = "Local Ollama"
base_url = "http://localhost:11434/v1/chat/completions"
auth = { secret = "" }
passthrough = true
[[combos]]
name = "smart-fallback"
description = "Cloud Groq first, local Ollama backup"
models = [
"groq/llama-3.3-70b-versatile",
"ollama/llama3.2"
]Validate your configuration:
./target/release/vibe-router config validate./target/release/vibe-router serve┌──────────────────────────────────────────────────────────┐
│ VIBE ROUTER v0.1.0 │
├──────────────────────────────────────────────────────────┤
│ Endpoint: http://127.0.0.1:20128/v1 │
│ Claude: http://127.0.0.1:20128/v1/messages │
│ Dashboard: http://127.0.0.1:20128/ │
│ Auth: Open (no API key required) │
│ Profile: workstation (RAM: 8192MB, Cores: 4) │
│ Providers: 2 │
│ Storage: vibe-router.db │
└──────────────────────────────────────────────────────────┘
Open http://localhost:20128 in your browser to access the dashboard.
Run Vibe Router and print the shell environment:
./target/release/vibe-router env claude-codeThen export the variables in your shell before launching claude:
export ANTHROPIC_BASE_URL="http://127.0.0.1:20128"
export ANTHROPIC_API_KEY="vibe-router"
claudefrom openai import OpenAI
client = OpenAI(
base_url="http://localhost:20128/v1",
api_key="none"
)
response = client.chat.completions.create(
model="smart-fallback", # Target a combo, alias, or provider/model
messages=[{"role": "user", "content": "Explain Rust borrowing in one sentence."}],
stream=True
)
for chunk in response:
content = chunk.choices[0].delta.content or ""
print(content, end="", flush=True)To connect Vibe Router as an MCP tool server for Claude Desktop, Cursor, or Hermes:
{
"mcpServers": {
"vibe-router": {
"command": "/path/to/vibe-router",
"args": ["mcp", "--config", "/path/to/vibe-router.toml"]
}
}
}Both features are configured under [server] and are off-the-shelf optional:
[server]
# Discover Vibe Local worker nodes on this PC and the LAN (UDP 20131 + HTTP 20129).
auto_discover_local = true
# Sync a Vibe_Radar model catalog and auto-build the "daily-free" combo (alias "free").
remote_catalog_url = "https://example.com/vibe-radar/catalog.json"
remote_catalog_interval_secs = 900 # 15 minutes; clamped to a 30s floor-
VIBE_LOCAL_TOKEN(environment variable): when set, discovery attaches it asAuthorization: BearerandX-Vibe-Local-Tokenon the model probe, and registered nodes reference it as anenv:secret resolved at request time. It is never written to disk or stored literally in the registry. -
Catalog feed shape (
Vibe_Radar):{ "updated_at": "2026-11-18T00:00:00Z", "providers": [ { "id": "catpub", "base_url": "https://api.catpub.example/v1/chat/completions", "models": [{ "id": "m1", "context": 8000 }] } ], "daily_free_chain": ["catpub/m1"] }Existing providers keep their configured endpoint and credentials — the catalog only merges new model metadata. New providers must use a public
http(s)host (loopback, link-local, and RFC1918 targets are rejected). Target the auto-built chain with the model namedaily-freeor its aliasfree.
Commands:
serve Start the OpenAI/Anthropic proxy server and dashboard (default)
config Inspect and validate configuration (e.g., config validate)
providers Inspect upstream providers and manage key pools
models List all addressable models, combos, and aliases
combos List configured combos and fallback chains
status Display usage and storage status
costs Display USD cost breakdown and latency metrics (p50/p95/p99)
alias Manage virtual model aliases (set, list)
proxy Manage tagged proxies (add, list)
relay Cloudflare Worker reverse-proxy relay utilities (worker-script)
env Print client environment variables (claude-code)
tui Live terminal status dashboard (alias: top)
mcp Model Context Protocol (MCP) stdio JSON-RPC 2.0 server
cache Inspect and purge in-memory prompt cache (stats, purge)
Vibe Router maintains 100% passing tests across all crates with zero warnings:
cargo test --workspace --all-features
cargo clippy --workspace --all-targets --all-features -- -D warnings
cargo fmt --all -- --check
cargo build --release --workspaceAll integration tests are hermetic and deterministic using an in-process TCP mock upstream that tests real HTTP sockets, chunked SSE streaming, timeouts, disconnects, failover chains, rate limits, and error status classification without relying on external networks or live accounts.
Licensed under the Apache License, Version 2.0 (the "License"); you may not use this file except in compliance with the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License.