Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions .dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,44 @@
# Keep local-only material out of the public image AND out of the build
# context upload. `COPY docs/ docs/` (Dockerfile) copies from the context
# with no knowledge of .gitignore, so every gitignored-or-untracked local
# doc would otherwise be baked into the released image — that is exactly
# how the tracked-release cleanliness gate went red inside the container
# (docs/WORKING_STATE.md) while staying green in a git-aware checkout.

# VCS + local tooling state (never image material; huge context otherwise)
.git
.venv
.claude
**/__pycache__
*.pyc

# Runtime/workspace outputs
outputs/
artifacts/
qf-demo/

# Local research + coordination material (mirrors .gitignore docs rules)
worldquant/
docs/WORKING_STATE.md
docs/quant_forge_overall_architecture.*
docs/backtest_rebuild/
docs/rd_evaluation_rebuild/
docs/*_repair_ledger_*.md
docs/first_principles_review_*.md

# Untracked local drafts living under COPY'd dirs
docs/getting_started_zh.md
docs/research_platform_optimization_from_vibe_quantgpt.md

# Local machine setup material at the context root
mac_b_setup/
dropbox_exclude.sh
PROJECT_STATUS.md
TWO_MAC_SYNC.md

# Local secrets and machine-local configs must never enter an image
configs/*.local.yaml
configs/*.local.env
configs/*.secrets.env
*.local.env
*.secrets.env
29 changes: 29 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -352,6 +352,35 @@ features. RD does not have a second API-key setting.

LLM 语义解析和 RD LLM 功能共用同一个当前 provider;RD 不再单独配置第二套 API key。

### Switching providers from the Web UI / 在网页端切换大模型

The workbench can also register a provider and take an API key at runtime, so
you can try a model without editing YAML or restarting the server. In the
expert-mode control rail, the **LLM Provider** menu lists the configured
providers plus the built-in presets (`deepseek`, `openai`, `openai_compatible`,
`glm`, `claude`, `minimax`). Switch **LLM API Key** to *前端输入 / manual*, enter
the key (and, for a preset with no default, the model / base URL), and click
**保存并启用 / Save & enable**.

The key is injected into the **running server process environment only**. It is
never written to disk, never echoed in any response, and never logged; a restart
clears it. For a durable key, keep using the ignored `configs/*.local.env` path
above. Like every mutating Web action, the settings write requires the browser
control token on a Docker/`0.0.0.0` bind. Pointing an existing provider at a new
`base_url` requires re-supplying the key in the same request, so a standing key
is never sent to a newly specified host.

网页端也可以在运行时注册 provider 并接收 API key,无需改 YAML 或重启即可试用某个
模型。在专家模式左侧控制栏,**LLM Provider** 下拉会列出已配置 provider 以及内置预设
(`deepseek`、`openai`、`openai_compatible`、`glm`、`claude`、`minimax`)。把
**LLM API Key** 切到「前端输入」,填入密钥(预设无默认值时再补填模型名 / base URL),
点击「保存并启用」。

密钥只注入**运行中的服务进程环境变量**:不写磁盘、不在任何响应中回显、不进日志,重启即
失效;如需持久保存,仍走上文被忽略的 `configs/*.local.env`。与所有会改状态的 Web 操作
一样,在 Docker / `0.0.0.0` 绑定下该写入需要浏览器控制令牌。把已有 provider 指向新的
`base_url` 时必须在同一请求内重新提交密钥,以确保常驻密钥不会被发往新指定的地址。

## Configuration Files / 配置文件

| File | Purpose |
Expand Down
6 changes: 4 additions & 2 deletions docs/full_integration_test_prompt.md
Original file line number Diff line number Diff line change
Expand Up @@ -152,8 +152,10 @@ provider 已识别、key env 已继承但真实 key 未打印、各数据/因子
- LLM API Key 控件(`#llm-api-key-mode`):
- “配置文件 / 环境变量加载”模式 → 输入框 `#llm-api-key` 置灰(disabled),**不显示
真实 key**;
- “手动输入(仅前端联调)”模式 → 输入框才可编辑;手动输入内容**仅前端使用、不持久化**,
绝不写入配置、日志、artifact 或报告(`data-secret-policy="not-submitted"`)。
- “前端输入(注入本次运行)”模式 → 输入框才可编辑;点「保存并启用」(`#llm-settings-save`)
经 `POST /api/settings/llm` 注入服务进程环境变量,**仅存内存、不落盘、任何响应不回显**,
重启后失效(`data-secret-policy="runtime-memory-only"`);内置预设(openai/glm/claude/
minimax/openai_compatible)可在此注册启用,缺省 model 的预设需补填模型名。
- 控制令牌门控 UX(token-gating):需要令牌时页面提示明确,粘贴/预置后功能解锁。

### 4.2 单因子研究 / single-factor module(真实 DeepSeek)
Expand Down
7 changes: 5 additions & 2 deletions docs/integration_workflow.md
Original file line number Diff line number Diff line change
Expand Up @@ -334,8 +334,11 @@ API/HTTP 调用只允许核对后端 contract、artifact 路径、日期窗口
- 默认模式应为“配置文件 / 环境变量加载”。
- 如果 provider key 已通过配置和环境变量加载,password 输入框必须置灰。
- 页面不得显示真实 API key,只能显示环境变量名或状态说明。
- 切换到“手动输入(仅前端联调)”时,输入框才可编辑。
- 手动输入的内容不得进入 parse/validate 请求体、日志、artifact 或报告;正式调用仍使用后端配置的环境变量。
- 切换到“前端输入(注入本次运行)”时,输入框才可编辑;点击“保存并启用”后密钥经
`POST /api/settings/llm`(受控制令牌门控)注入服务进程环境变量,即时生效。
- 密钥仅存在于运行中进程的内存,不写入磁盘、配置、日志、artifact 或报告,任何响应
(含 `/api/status`)都不得回显密钥值;重启后失效,持久化仍走 `configs/*.local.env`。
- 手动输入的密钥不得进入 parse/validate 请求体;正式调用仍从后端环境变量读取。
5. 输入自然语言因子观点,点击“解析因子”。
6. 解析完成后应只展示 factor 草稿和待确认参数,不应立即执行评价/回测。
7. 检查评测参数区是否自动填入默认值:
Expand Down
203 changes: 201 additions & 2 deletions src/quant_forge/apps/web/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@
import re
import threading
from typing import Any
from urllib.parse import parse_qs, unquote
from urllib.parse import parse_qs, unquote, urlparse

from quant_forge.apps.web.jobs import _IdeaValidationSettings, _WebJobCancelled, _client_error_message
from quant_forge.apps.web.markdown import extract_markdown_title, render_markdown_html
Expand All @@ -29,7 +29,12 @@
_staggered_warnings_count,
)
from quant_forge.backtesting.service import EXTERNAL_OOS_ROLE
from quant_forge.config import QuantForgeConfig, simulation_profile_from_mapping, validate_llm_runtime
from quant_forge.config import (
QuantForgeConfig,
llm_settings_with_provider_update,
simulation_profile_from_mapping,
validate_llm_runtime,
)
from quant_forge.core.contracts import (
BacktestResult,
EvaluationResult,
Expand Down Expand Up @@ -2435,13 +2440,37 @@ def _paths_payload(config: QuantForgeConfig) -> dict[str, str]:


def _llm_provider_options(config: QuantForgeConfig) -> tuple[dict[str, str], ...]:
from quant_forge.llm_client import builtin_provider_presets

options: list[dict[str, str]] = []
configured_names: set[str] = set()
for option in config.llm.public_provider_options():
configured_names.add(option["provider"])
runtime_ready, runtime_error = _llm_runtime_status(config, option["provider"])
enriched = dict(option)
enriched["configured"] = "true"
enriched["runtime_ready"] = "true" if runtime_ready else "false"
enriched["runtime_error"] = runtime_error
options.append(enriched)
# Built-in presets not declared in YAML: offered so the frontend can
# register them at runtime via /api/settings/llm without a config edit.
# Only metadata travels here — the env var NAME, never key material.
for preset in builtin_provider_presets():
if preset["provider"] in configured_names:
continue
key_present = bool(os.environ.get(preset["api_key_env"], ""))
options.append(
{
**preset,
"configured": "false",
"runtime_ready": "false",
"runtime_error": (
"preset is not enabled yet; save LLM settings to enable it"
if key_present
else "preset is not enabled yet; save LLM settings with an API key to enable it"
),
}
)
return tuple(options)


Expand All @@ -2457,6 +2486,176 @@ def _active_llm(config: QuantForgeConfig) -> Any:
return config.llm.select_provider()


_LLM_API_KEY_MAX_CHARS = 512


def apply_llm_settings_update(
config: QuantForgeConfig, payload: dict[str, Any]
) -> tuple[QuantForgeConfig, dict[str, Any]]:
"""Apply a runtime LLM settings update from ``POST /api/settings/llm``.

Semantics (all in-process, nothing is written to disk):

* ``provider`` (required) — a YAML-configured provider or a built-in
preset; presets are registered into the runtime config on first save.
* ``model`` / ``base_url`` (optional) — override or complete the entry;
the merged entry passes the same validation as YAML-loaded providers.
* ``api_key`` (optional) — injected into ``os.environ[api_key_env]`` so
the very next LLM call picks it up (the client reads the key from the
environment per call). The key value is never echoed, logged, or
persisted; a restart clears it. For a durable key, users keep the
documented path: a git-ignored ``configs/*.local.env``.
* ``activate`` (default true) — make this provider the active default
(used by RD optimization and the sidecar readiness probe; parse jobs
already choose per request).

Security: this is the only route that could turn workbench-operate access
into standing-secret exfiltration, so it refuses to point an existing
provider whose key already lives in the environment at a NEW ``base_url``
without the caller re-supplying the key in the SAME request. Otherwise a
caller could redirect, say, deepseek to ``http://attacker/v1`` with no key
and the next LLM call would ship the host's durable ``DEEPSEEK_API_KEY``
(from ``configs/*.local.env``) as a bearer token to the attacker's URL —
a credential the web layer is never allowed to reveal. Legitimate
first-time setup (host + key together) is unaffected. No private-IP/SSRF
blocklist is applied on purpose: a local self-hosted model
(``http://127.0.0.1:11434`` etc.) is a first-class OpenAI-compatible use.

Returns the replacement frozen config plus a response payload shaped like
the ``llm`` section of ``/api/status`` so the frontend can re-hydrate from
the response alone (loopback binds never re-fetch ``/api/status``).
"""

from quant_forge.llm_client import builtin_provider_presets, canonical_provider_name

provider_raw = str(payload.get("provider", "") or "").strip()
if not provider_raw:
raise ValueError("provider is required")
name = canonical_provider_name(provider_raw)
if name in {"rule", "deterministic"}:
raise ValueError("provider 'rule' is built in and cannot be edited")

model = str(payload.get("model", "") or "").strip()
base_url = str(payload.get("base_url", "") or "").strip()
activate = _bool_parameter(payload.get("activate", True), "activate")
api_key = payload.get("api_key")
api_key_supplied = api_key is not None and str(api_key) != ""

presets = {preset["provider"]: preset for preset in builtin_provider_presets()}
api_key_env = ""
existing = config.llm.providers.get(name)
if existing is None:
preset = presets.get(name)
if preset is None:
known = ", ".join(sorted(set(config.llm.providers) | set(presets)))
raise ValueError(f"unknown LLM provider: {provider_raw}. Known providers: {known}.")
model = model or preset["model"]
base_url = base_url or preset["base_url"]
api_key_env = preset["api_key_env"]

if base_url:
_require_http_base_url(base_url)

new_llm = llm_settings_with_provider_update(
config.llm,
provider=name,
model=model,
base_url=base_url,
api_key_env=api_key_env,
activate=activate,
)
selected = new_llm.select_provider(name)

# Standing-key exfiltration guard (see the security note above): a base_url
# that actually changes the entry's host, without a key re-supplied here,
# is refused when the target env var is already populated. `base_url` is
# only truthy when the caller sent one (presets fill it before this point,
# but a brand-new preset registration has no standing key + no `existing`).
target_env = str(selected.api_key_env).strip()
prior_base_url = existing.base_url if existing is not None else ""
if (
base_url
and base_url != prior_base_url
and not api_key_supplied
and target_env
and os.environ.get(target_env)
):
raise ValueError(
"changing base_url requires re-supplying api_key so a standing "
"credential is never sent to a newly specified host"
)

key_updated = False
if api_key_supplied:
credential = str(api_key)
_validate_api_key_shape(credential)
env_name = target_env
if not env_name:
raise ValueError(
f"provider {name} does not declare api_key_env; cannot accept an API key"
)
_validate_env_var_name(env_name)
# In-process only, by design: the LLM client resolves the key from the
# environment on every call, so this takes effect immediately and
# vanishes on restart. Never logged, never in any response payload.
os.environ[env_name] = credential
key_updated = True

new_config = replace(config, llm=new_llm)
active = new_config.llm.select_provider()
response = {
"ok": True,
"key_updated": key_updated,
"llm": {
"provider": active.provider,
"model": active.model,
"api_key_env": active.api_key_env,
"providers": _llm_provider_options(new_config),
},
}
return new_config, response


def _validate_env_var_name(env_name: str) -> None:
"""Refuse to write anything but a plain env-var identifier.

Presets use fixed ``*_API_KEY`` names and YAML is owner-controlled, so this
is defense-in-depth: it stops a future/hand-edited ``api_key_env`` from
aiming the runtime key write at a process-influencing variable
(``PATH``/``LD_PRELOAD``/``PYTHONPATH``/the control-token env). Mirrors the
identifier rule the env-*file* parser already enforces (config.py).
"""

if not re.fullmatch(r"[A-Za-z_][A-Za-z0-9_]*", env_name):
raise ValueError("api_key_env must be a plain environment-variable name")
if env_name.endswith(("PATH", "PRELOAD")) or env_name in {"PATH", "LD_PRELOAD", "PYTHONPATH"}:
raise ValueError(f"refusing to write reserved environment variable: {env_name}")


def _validate_api_key_shape(credential: str) -> None:
"""Shape-only checks; error text never includes the submitted value."""

if len(credential) > _LLM_API_KEY_MAX_CHARS:
raise ValueError(f"api_key is too long (max {_LLM_API_KEY_MAX_CHARS} characters)")
if any(character.isspace() for character in credential):
raise ValueError("api_key must not contain whitespace")
if any(not character.isprintable() for character in credential):
raise ValueError("api_key must not contain non-printable characters")


def _require_http_base_url(base_url: str) -> None:
"""The web write path only accepts http(s) endpoints.

``urllib.request.urlopen`` would happily open other schemes (``file://``
included), so a runtime-writable base_url must be scheme-restricted here
even though YAML-configured values are the host owner's own business.
"""

scheme = urlparse(base_url).scheme.lower()
if scheme not in {"http", "https"}:
raise ValueError("base_url must start with http:// or https://")


def _rd_status_payload(config: QuantForgeConfig, rd_config: ResearchLoopConfig) -> dict[str, str]:
active_llm = _active_llm(config)
return {
Expand Down
20 changes: 18 additions & 2 deletions src/quant_forge/apps/web/html.py
Original file line number Diff line number Diff line change
Expand Up @@ -44,8 +44,14 @@ def _provider_options_script_payload(options: tuple[dict[str, str], ...]) -> tup
return tuple(
{
"provider": option.get("provider", ""),
"model": option.get("model", ""),
# boolean flag only: a real URL literal would break the D8
# no-external-reference sweep on the served page, and the
# frontend only needs to know whether base_url must be typed in.
"hasBaseUrl": "true" if option.get("base_url", "") else "false",
"apiKeyEnv": option.get("api_key_env", ""),
"runtimeReady": option.get("runtime_ready", "false"),
"configured": option.get("configured", "true"),
}
for option in options
)
Expand All @@ -65,6 +71,8 @@ def _page_config_json(


def _provider_readiness_label(option: dict[str, str]) -> str:
if option.get("configured") == "false":
return " · 预设未启用"
if option.get("runtime_ready") == "true":
return " · env " + option["api_key_env"] if option["api_key_env"] else " · no auth"
api_key_env = option.get("api_key_env", "")
Expand Down Expand Up @@ -469,6 +477,9 @@ def _index_html(
background: #fff6f6;
}}
button:disabled {{ opacity: .55; cursor: wait; }}
/* label{{display:block}} / button width rules above would defeat the
[hidden] UA default on the runtime LLM settings controls. */
label[hidden], input[hidden], button[hidden] {{ display: none; }}
code {{
background: #eef5ef;
border: 1px solid var(--line);
Expand Down Expand Up @@ -1165,9 +1176,14 @@ def _index_html(
<label for="llm-api-key-mode">LLM API Key</label>
<select id="llm-api-key-mode">
<option value="config">配置文件 / 环境变量加载</option>
<option value="manual">手动输入(仅前端联调)</option>
<option value="manual">前端输入(注入本次运行)</option>
</select>
<input id="llm-api-key" type="password" autocomplete="off" data-secret-policy="not-submitted" disabled>
<input id="llm-api-key" type="password" autocomplete="off" data-secret-policy="runtime-memory-only" disabled>
<label for="llm-model" id="llm-model-label" hidden>模型名称</label>
<input id="llm-model" autocomplete="off" hidden>
<label for="llm-base-url" id="llm-base-url-label" hidden>Base URL</label>
<input id="llm-base-url" autocomplete="off" hidden>
<button id="llm-settings-save" class="secondary" hidden>保存并启用</button>
<p id="llm-api-key-status" class="meta"></p>
<!-- P1 (agent_sidecar_frontend.md §5.1/§8): the resident 11-parameter
grid (formerly #validation-controls, wrapped in #advanced-params)
Expand Down
Loading
Loading