feat(local): OpenRouter provider (BYO key)

plans/community-local/00-umbrella-plan.md

@@ -84,7 +84,7 @@ Docker Compose, Postgres, Zero, Redis, Celery, LangGraph, git KB, scrapers, MCP,
 | **HTTP stack** | **FastAPI** + uvicorn | Same stack as cloud backend; native OpenAPI for frontend; SSE streaming for chat. PyInstaller risk is handled in [`api/00-spike.md`](api/00-spike.md) — not a reason to downgrade. |
 | **Retrieval** | **FTS5 + sqlite-vec hybrid, cosine rescore** | Semantic + keyword from day one: both legs widen recall, cosine orders. Embeddings on ingest must be queried properly — not keyword-only, not in-memory scan over BLOBs. |
 | **Embed provider** | Bundled bge-small-en-v1.5 int8, in-process on onnxruntime | 384-dim, ~66MB, runs offline on CPU with no model server. Docling parses, Chonkie chunks. Remote embedding is a later opt-in, not a launch dependency. |
-| **Generation provider** | Ollama default; curated Qwen catalog; provider `Protocol`s | `modules/llm/`. A `Generator` answers, a `ModelStore` downloads; a remote API satisfies only the first, so the download UI is gated by `isinstance`, not a provider name. Ollama has no library API, so the offered models are a curated Qwen list inside the Ollama provider. `SelectedModel(role)` holds the choice. Adding a provider is a folder plus one registry line. |
+| **Generation provider** | Ollama default; curated Qwen catalog; OpenRouter for BYO-key remote models; provider `Protocol`s | `modules/llm/`. A `Generator` answers, a `ModelStore` downloads; a remote API satisfies only the first, so the download UI is gated by `isinstance`, not a provider name. Ollama has no library API, so the offered models are a curated Qwen list inside the Ollama provider. OpenRouter is the hosted option for models too large to run locally: the user brings their own key, stored plaintext in the local DB (`provider_credentials`; ponytail ceiling — OS keyring is the upgrade path), and the key-entry UI is gated by a `requires_key`/`configured` flag. `SelectedModel(role)` holds the choice. Adding a provider is a folder plus one registry line. |
 | **Ollama runtime** | **Bundled as a supervised sidecar (packaged); dev uses the developer's own `ollama serve`** | Chat can't depend on a daemon the user may not have installed. The packaged app ships the standalone Ollama archive (`electron/scripts/fetch-ollama.mjs` stages it, electron-builder carries it in `resources/ollama`) and Electron runs it as a third sidecar on a chosen port, passing `SURFSENSE_LOCAL_OLLAMA_BASE_URL` to the API. Models aren't shipped — the user pulls into the writable data dir after install. The two Python sidecars and Ollama share one supervisor (`electron/src/main/sidecars/`): the supervisor spawns/reaps, one spec file per sidecar carries its identity. Ollama is best-effort at boot (only the API gates the window; its state surfaces via `/llm/providers`). |
 | **Persistence** | SQLAlchemy 2.0 + Alembic, same as cloud | Models are the source of truth. `versions/` ships as PyInstaller data, resolved from the package's own `__file__` — de-risked in [`api/00-spike.md`](api/00-spike.md). |
 | **Migrations** | **Hand-written; autogenerate is off** | Autogenerate cannot see a rename — it emits drop + add, which deletes a column's data silently. The target database is one user's laptop, unbacked and uninspectable, so every revision is written and read by a person. `env.py` carries no `target_metadata`, so `--autogenerate` cannot be used by accident. Mature SQLite-backed apps make the same call — hand-written revisions throughout. |

surfsense_local/backend/alembic/versions/0001_initial_schema.py

@@ -235,6 +235,15 @@ def upgrade() -> None:
         sa.PrimaryKeyConstraint("role", name=op.f("pk_selected_models")),
     )
 
+    op.create_table(
+        "provider_credentials",
+        sa.Column("provider", sa.String(), nullable=False),
+        sa.Column("api_key", sa.String(), nullable=False),
+        sa.Column("updated_at", sa.DateTime(), server_default=NOW, nullable=False),
+        # One BYO key per provider; the provider name keys the row.
+        sa.PrimaryKeyConstraint("provider", name=op.f("pk_provider_credentials")),
+    )
+
     op.create_table(
         "artifact_files",
         sa.Column("id", sa.Integer(), nullable=False),
@@ -267,6 +276,7 @@ def upgrade() -> None:
 
 def downgrade() -> None:
     """Drop in dependency order; SQLite refuses to drop a referenced parent."""
+    op.drop_table("provider_credentials")
     op.drop_table("selected_models")
     op.drop_table("artifact_files")
     op.drop_table("artifacts")

surfsense_local/backend/modules/chat/router.py

@@ -88,7 +88,7 @@ async def send_message(
     selected = session.get(SelectedModel, ModelRole.GENERATION)
     if selected is None:
         raise HTTPException(status.HTTP_409_CONFLICT, "no chat model selected")
-    generator = get_provider(selected.provider)
+    generator = get_provider(selected.provider, session)
     if generator is None:
         raise HTTPException(
             status.HTTP_409_CONFLICT, f"unknown provider: {selected.provider}"

surfsense_local/backend/modules/llm/credentials.py

@@ -0,0 +1,29 @@
+"""BYO provider API keys, kept in the local database (see ProviderCredential)."""
+
+from sqlalchemy.orm import Session
+
+from modules.llm.models import ProviderCredential
+
+
+def read_provider_key(session: Session, provider: str) -> str | None:
+    row = session.get(ProviderCredential, provider)
+    return row.api_key if row else None
+
+
+def write_provider_key(session: Session, provider: str, api_key: str) -> None:
+    row = session.get(ProviderCredential, provider)
+    if row is None:
+        session.add(ProviderCredential(provider=provider, api_key=api_key))
+    else:
+        row.api_key = api_key
+    session.flush()
+
+
+def clear_provider_key(session: Session, provider: str) -> bool:
+    """Remove the key if present; returns whether there was one."""
+    row = session.get(ProviderCredential, provider)
+    if row is None:
+        return False
+    session.delete(row)
+    session.flush()
+    return True

surfsense_local/backend/modules/llm/dependencies.py

@@ -2,13 +2,14 @@
 
 from fastapi import Depends, HTTPException, status
 
+from api.dependencies import SessionDep
 from modules.llm.providers import get_provider
 from modules.llm.providers.protocols import Generator, ModelStore
 
 
-def get_provider_or_404(provider: str) -> Generator:
+def get_provider_or_404(provider: str, session: SessionDep) -> Generator:
     """Resolve the provider in the path, or fail before the handler."""
-    found = get_provider(provider)
+    found = get_provider(provider, session)
     if found is None:
         raise HTTPException(status.HTTP_404_NOT_FOUND, f"unknown provider: {provider}")
     return found

surfsense_local/backend/modules/llm/models.py

@@ -21,3 +21,16 @@ class SelectedModel(Base):
     updated_at: Mapped[datetime] = mapped_column(
         server_default=func.now(), onupdate=func.now()
     )
+
+
+class ProviderCredential(Base):
+    __tablename__ = "provider_credentials"
+
+    # One key per provider (BYO); the provider name is the key.
+    # ponytail: plaintext — the db is one user's local file. Upgrade path: hold
+    # the secret in the OS keyring and keep only a presence flag here.
+    provider: Mapped[str] = mapped_column(primary_key=True)
+    api_key: Mapped[str]
+    updated_at: Mapped[datetime] = mapped_column(
+        server_default=func.now(), onupdate=func.now()
+    )

surfsense_local/backend/modules/llm/providers/__init__.py

@@ -1,20 +1,35 @@
 from collections.abc import Callable
 
+from sqlalchemy.orm import Session
+
+from modules.llm.credentials import read_provider_key
 from modules.llm.providers.ollama.provider import OllamaProvider
+from modules.llm.providers.openrouter.provider import OpenRouterProvider
 from modules.llm.providers.protocols import Generator
 from shared.config import get_llm_settings
 
 # Name to provider. Adding one is a new folder and one line here, never a change
 # to a consumer: the router resolves everything through get_provider().
 REGISTRY: dict[str, Callable[[], Generator]] = {
     "ollama": lambda: OllamaProvider(get_llm_settings().ollama_base_url),
+    "openrouter": lambda: OpenRouterProvider(get_llm_settings().openrouter_base_url),
 }
 
 
 def provider_names() -> list[str]:
     return list(REGISTRY)
 
 
-def get_provider(name: str) -> Generator | None:
+def get_provider(name: str, session: Session | None = None) -> Generator | None:
+    """Build the provider, and give a BYO-key one its key from the database.
+
+    The key lookup needs a session; callers that have one (every router handler)
+    pass it, so a keyed provider comes back ready to answer.
+    """
     factory = REGISTRY.get(name)
-    return factory() if factory else None
+    if factory is None:
+        return None
+    provider = factory()
+    if session is not None and getattr(provider, "requires_key", False):
+        provider.api_key = read_provider_key(session, provider.name)
+    return provider

surfsense_local/backend/modules/llm/providers/ollama/provider.py

@@ -15,6 +15,7 @@ class OllamaProvider:
     """Ollama over its native API: it both answers and holds models on disk."""
 
     name = "ollama"
+    requires_key = False
 
     def __init__(self, base_url: str) -> None:
         self._base_url = base_url.rstrip("/")

surfsense_local/backend/modules/llm/providers/openrouter/provider.py

@@ -0,0 +1,91 @@
+import json
+from collections.abc import AsyncIterator
+
+import httpx
+
+from modules.llm.providers.types import Message, Model
+
+# Generation streams for a while; auth and model listing are quick.
+TIMEOUT = httpx.Timeout(120.0, connect=5.0)
+
+
+class OpenRouterProvider:
+    """OpenRouter over its OpenAI-compatible API: a hosted generator, BYO key.
+
+    Holds no models on disk, so it is a Generator but not a ModelStore — the
+    download UI stays hidden. The key is set on the instance by the registry
+    from the database; without one it reports unhealthy and lists nothing.
+    """
+
+    name = "openrouter"
+    requires_key = True
+
+    def __init__(self, base_url: str, api_key: str | None = None) -> None:
+        self._base_url = base_url.rstrip("/")
+        self.api_key = api_key
+
+    def _client(self) -> httpx.AsyncClient:
+        headers = {"X-Title": "SurfSense"}  # OpenRouter attribution, optional
+        if self.api_key:
+            headers["Authorization"] = f"Bearer {self.api_key}"
+        return httpx.AsyncClient(
+            base_url=self._base_url, timeout=TIMEOUT, headers=headers
+        )
+
+    async def health(self) -> bool:
+        if not self.api_key:
+            return False
+        try:
+            async with self._client() as client:
+                return (await client.get("/key")).status_code == 200
+        except httpx.HTTPError:
+            return False
+
+    async def models(self) -> list[Model]:
+        if not self.api_key:
+            return []
+        async with self._client() as client:
+            reply = await client.get("/models")
+            reply.raise_for_status()
+            entries = reply.json().get("data", [])
+
+        return [
+            Model(entry["id"], installed=True, capabilities=("completion",))
+            for entry in entries
+            if _answers_text(entry)
+        ]
+
+    async def chat(self, model: str, messages: list[Message]) -> AsyncIterator[str]:
+        body = {
+            "model": model,
+            "messages": [{"role": m.role, "content": m.content} for m in messages],
+            "stream": True,
+        }
+        async with (
+            self._client() as client,
+            client.stream("POST", "/chat/completions", json=body) as reply,
+        ):
+            reply.raise_for_status()
+            async for line in reply.aiter_lines():
+                delta = _delta(line)
+                if delta:
+                    yield delta
+
+
+def _answers_text(entry: dict) -> bool:
+    """Keep text-out (chat) models; skip image- or embedding-only endpoints."""
+    modalities = entry.get("architecture", {}).get("output_modalities")
+    return "text" in modalities if modalities else True
+
+
+def _delta(line: str) -> str | None:
+    """One token from an OpenAI SSE line; None for keep-alives and `[DONE]`."""
+    if not line.startswith("data:"):
+        return None
+    payload = line[len("data:") :].strip()
+    if not payload or payload == "[DONE]":
+        return None
+    choices = json.loads(payload).get("choices")
+    if not choices:
+        return None
+    return choices[0].get("delta", {}).get("content")

surfsense_local/backend/modules/llm/router.py

@@ -1,16 +1,23 @@
 import json
 from collections.abc import AsyncIterator
 
-from fastapi import APIRouter, HTTPException, status
+from fastapi import APIRouter, HTTPException, Response, status
 from fastapi.responses import StreamingResponse
 
 from api.dependencies import SessionDep
+from modules.llm.credentials import (
+    clear_provider_key,
+    read_provider_key,
+    write_provider_key,
+)
 from modules.llm.dependencies import ProviderDep, StoreDep
 from modules.llm.models import ModelRole, SelectedModel
 from modules.llm.providers import get_provider, provider_names
 from modules.llm.providers.protocols import ModelStore
 from modules.llm.schemas import (
     CatalogEntryRead,
+    CredentialStatus,
+    CredentialWrite,
     ModelRead,
     ProviderRead,
     PullRequest,
@@ -22,13 +29,16 @@
 
 
 @router.get("/providers", response_model=list[ProviderRead], summary="List providers")
-async def list_providers() -> list[ProviderRead]:
-    providers = [get_provider(name) for name in provider_names()]
+async def list_providers(session: SessionDep) -> list[ProviderRead]:
+    providers = [get_provider(name, session) for name in provider_names()]
     return [
         ProviderRead(
             name=provider.name,
             healthy=await provider.health(),
             can_download=isinstance(provider, ModelStore),
+            requires_key=getattr(provider, "requires_key", False),
+            configured=not getattr(provider, "requires_key", False)
+            or read_provider_key(session, provider.name) is not None,
         )
         for provider in providers
         if provider is not None
@@ -91,6 +101,42 @@ async def progress() -> AsyncIterator[bytes]:
     return StreamingResponse(progress(), media_type="application/x-ndjson")
 
 
+@router.put(
+    "/providers/{provider}/credentials",
+    response_model=CredentialStatus,
+    summary="Set a provider's API key",
+)
+def set_credential(
+    provider: str, payload: CredentialWrite, session: SessionDep
+) -> CredentialStatus:
+    found = get_provider(provider)
+    if found is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, f"unknown provider: {provider}")
+    if not getattr(found, "requires_key", False):
+        raise HTTPException(status.HTTP_409_CONFLICT, f"{provider} needs no API key")
+
+    key = payload.api_key.strip()
+    if not key:
+        raise HTTPException(
+            status.HTTP_422_UNPROCESSABLE_CONTENT, "api key must not be empty"
+        )
+
+    write_provider_key(session, provider, key)
+    return CredentialStatus(provider=provider, configured=True)
+
+
+@router.delete(
+    "/providers/{provider}/credentials",
+    status_code=status.HTTP_204_NO_CONTENT,
+    summary="Remove a provider's API key",
+)
+def clear_credential(provider: str, session: SessionDep) -> Response:
+    if get_provider(provider) is None:
+        raise HTTPException(status.HTTP_404_NOT_FOUND, f"unknown provider: {provider}")
+    clear_provider_key(session, provider)
+    return Response(status_code=status.HTTP_204_NO_CONTENT)
+
+
 @router.get(
     "/selection/{role}",
     response_model=SelectionRead,
@@ -112,7 +158,7 @@ def read_selection(role: ModelRole, session: SessionDep) -> SelectedModel:
 async def set_selection(
     role: ModelRole, payload: SelectionWrite, session: SessionDep
 ) -> SelectedModel:
-    provider = get_provider(payload.provider)
+    provider = get_provider(payload.provider, session)
     if provider is None:
         raise HTTPException(
             status.HTTP_422_UNPROCESSABLE_CONTENT,

surfsense_local/backend/modules/llm/schemas.py

@@ -11,6 +11,10 @@ class ProviderRead(BaseModel):
     name: str
     healthy: bool
     can_download: bool
+    # BYO-key providers need a key before they answer; the UI shows a key field
+    # when one is required and not yet set.
+    requires_key: bool
+    configured: bool
 
 
 class ModelRead(BaseModel):
@@ -36,6 +40,19 @@ class PullRequest(BaseModel):
     name: str
 
 
+class CredentialWrite(BaseModel):
+    """The BYO API key a client sets for a provider."""
+
+    api_key: str
+
+
+class CredentialStatus(BaseModel):
+    """Whether a provider has a key on file. The key itself is never returned."""
+
+    provider: str
+    configured: bool
+
+
 class SelectionWrite(BaseModel):
     """The choice a client makes for a role."""
 

surfsense_local/backend/shared/config.py

@@ -64,6 +64,10 @@ class LLMSettings(BaseSettings):
     # Electron runs Ollama on a port it chose and sets the env var.
     ollama_base_url: str = "http://127.0.0.1:11434"
 
+    # OpenRouter is the hosted, BYO-key option; the key lives in the database,
+    # not here. Overridable only so tests can point at a stub.
+    openrouter_base_url: str = "https://openrouter.ai/api/v1"
+
 
 @lru_cache
 def get_storage_settings() -> StorageSettings: