Skip to content

feat(web): runtime LLM provider switching + API key intake from the frontend - #22

Merged
agenthaulk merged 4 commits into
mainfrom
feature/web-llm-settings
Jul 17, 2026
Merged

agenthaulk merged 4 commits into
mainfrom
feature/web-llm-settings

Conversation

@agenthaulk

@agenthaulk agenthaulk commented Jul 17, 2026 •

Copy link
Copy Markdown
Owner

Summary

Adds a runtime LLM-provider switcher and API-key intake to the web workbench, so a user can pick a provider and supply its key from the frontend without editing YAML or restarting the server.

  • Provider menu now lists YAML-configured providers and the built-in presets (deepseek / openai / openai_compatible / glm / claude / minimax), each tagged with its readiness. Enabling a preset no longer needs a config edit.
  • POST /api/settings/llm (control-token gated like every other POST) registers/updates a provider and optionally injects an API key. activate flips the active default; the routing closure swaps its frozen config atomically under a lock and hands the replacement to the sidecar tool registry so every surface agrees.
  • Frontend: the previously inert key field becomes the intake path — password input, cleared on save, never stored client-side. Presets missing a model/base_url reveal the needed inputs; the runtime strip re-hydrates from the save response (loopback binds never re-fetch /api/status).

Security model

The API key is injected into the server process environment only (the client resolves keys from env per call, so it takes effect on the next request). It is never persisted, echoed, or logged — durable keys keep the documented gitignored configs/*.local.env path.

Adversarial review of the credential path found the submitted-key flow airtight (no leak to any response, log, disk, job, run index, sidecar journal, or served HTML; the env-injection-via-payload attack is unreachable) and surfaced one issue that this PR fixes:

  • base_url key-exfiltration (fixed). A web-writable base_url let a caller redirect an existing provider to an attacker URL with no key, so the next call would ship the host's standing key as a bearer token. Now a base_url change that moves an existing provider to a new host is refused unless the key is re-supplied in the same request — an attacker must bring their own (harmless) key. First-time preset setup (host+key together) is unaffected. No private-IP/SSRF blocklist is applied on purpose: a local self-hosted model (http://127.0.0.1:11434) is a first-class OpenAI-compatible endpoint.
  • The target env-var name is validated before the runtime write (plain identifier; PATH/LD_PRELOAD/PYTHONPATH refused).
  • The settings read-modify-write runs under a lock (no dropped registration on concurrent POSTs); /api/status snapshots the config once (no torn mid-swap read).

Build hygiene

Adds a .dockerignore. COPY docs/ copied from the build context with no knowledge of .gitignore, so gitignored local docs were being baked into the image and the tracked-release cleanliness gate went red inside the container (no .git there) while staying green in a checkout. The ignore file keeps local material, workspace outputs, and local secret configs out of the image context.

Docs

README.md gains a Switching providers from the Web UI subsection under LLM Provider Setup, and the integration checklists (docs/integration_workflow.md, docs/full_integration_test_prompt.md) are updated to the new runtime-memory-only key policy.

Tests / gates

  • New tests/test_web_llm_settings.py pins the security contract: no key in any response, no key on workspace disk, 401 without token, the 400 taxonomy, provider-alias mapping, preset-metadata-only exposure, the base_url exfiltration guard, the reserved-env-var refusal, and the concurrent-save no-drop invariant.
  • Full suite green locally (1989 passed) and in the rebuilt image (1976 passed, 13 skipped); release_safety_scan.py passes on 278 public files.
  • Verified end-to-end in a Docker container against the mounted data set and a real UI click-through: preset registration flips readiness, the guard blocks a keyless redirect, switching back to the configured provider leaves its standing key intact, and no key touches disk.

Quant Forge contributors added 4 commits July 16, 2026 20:48
…rontend

New POST /api/settings/llm (control-token gated like every POST):
- registers/updates a provider at runtime: YAML-configured entries and the
  built-in presets (deepseek/openai/glm/claude/minimax/openai_compatible)
  are both offered, so enabling a provider no longer requires a config edit;
- optional api_key is injected into the server process environment only
  (the client resolves keys from env per call, so it takes effect on the
  next request); the value is never persisted, logged, or echoed back --
  durable keys keep the documented gitignored configs/*.local.env path;
- activate switches the active default provider; the routing closure swaps
  its frozen config snapshot atomically and hands the replacement to the
  sidecar tool registry so every surface agrees;
- web-writable base_url is scheme-restricted to http(s) (urlopen would
  otherwise accept file://).

Frontend: the previously inert #llm-api-key input (not-submitted policy)
becomes the intake path -- password field, cleared on save, never stored
client-side; preset entries surface model/base_url inputs when the preset
has no default; runtime strip re-hydrates from the save response so
loopback binds (which never re-fetch /api/status) stay correct.

Tests: tests/test_web_llm_settings.py pins the security contract (no key
in any response, no key on workspace disk, 401 without token, 400 taxonomy,
alias mapping, preset metadata-only). Workbench pins updated to the new
data-secret-policy=runtime-memory-only stance; integration docs updated.
…ages

COPY docs/ copied from the build context with no knowledge of .gitignore,
so gitignored local session docs were baked into the public image; the
tracked-release cleanliness gate then went red inside the container (no
.git => the gitless fallback walk scans everything) while staying green
in a git-aware checkout. Mirror the local-doc ignore rules, exclude
workspace outputs, VCS state, and local secret configs from the context.
Adversarial review of the runtime LLM settings endpoint surfaced one MAJOR
issue and hardening items; all addressed:

- MAJOR standing-key exfiltration: base_url was web-writable, so a caller
  could redirect an existing provider (e.g. deepseek) to an attacker URL
  with NO api_key and the next LLM call would ship the host's durable
  DEEPSEEK_API_KEY as a bearer token. Now a base_url change that would move
  an existing provider to a new host is refused unless the key is
  re-supplied in the same request -- forcing an attacker to bring their own
  (harmless) key. First-time preset setup (host+key together) is unaffected.
  No private-IP/SSRF blocklist: a local self-hosted model (127.0.0.1:11434)
  is a first-class OpenAI-compatible endpoint and must keep working.
- env-var name is now validated before the runtime key write (plain
  identifier, and PATH/LD_PRELOAD/PYTHONPATH refused) so a hand-edited
  api_key_env can't aim the write at a process-influencing variable.
- the settings read-modify-write (env inject + config swap + registry
  handoff) now runs under a lock, so two concurrent POSTs can't drop a
  registration; the swap comment states the true snapshot semantics.
- /api/status snapshots the closure config once to avoid a torn mid-swap read.
- tests register every touched provider env var with monkeypatch (no host
  key clobbered); new tests pin the exfiltration guard, the reserved-env
  refusal, and the concurrent-save no-drop invariant.
Add a subsection under LLM Provider Setup covering the new
POST /api/settings/llm frontend path: the provider menu now offers the
built-in presets, the key is injected into the server process environment
only (never persisted/echoed/logged), the settings write is control-token
gated on a 0.0.0.0 bind, and a base_url change to an existing provider
requires re-supplying the key.
@agenthaulk
agenthaulk merged commit 7e847b4 into main Jul 17, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant