Skip to content

fix(parser): warn when an idea needs data outside the price-volume panel - #26

Open
agenthaulk wants to merge 2 commits into
mainfrom
fix/honest-unmappable-warning
Open

agenthaulk wants to merge 2 commits into
mainfrom
fix/honest-unmappable-warning

Conversation

@agenthaulk

Copy link
Copy Markdown
Owner

Problem

A fundamentals idea — e.g. 年报、季报利润同比增加高的公司,在未来21天的表现会更好 — parsed to a confident-looking rank(return_5d) with no warning.

Root cause is two deliberate constraints of the public workbench colliding: the panel has only price-volume fields (close/is_st/market_cap/return_/volatility_/volume — no fundamentals), and the parser system prompt allows only a handful of price-volume formulas. So an unmappable idea degrades to the nearest canned formula. The existing no-silent-fallback guard only tripped on the rank(close) catch-all, so a rank(return_5d) substitution slipped through looking confident.

Fix (lightweight, honesty-only)

  • specs/nl_flow.out_of_scope_data_warnings(text): a curated, conservative keyword detector for data domains the panel can't express (financial statements, valuation multiples, alt-data — Chinese + English), chosen to avoid colliding with in-scope price-volume vocabulary. The bare word 估值 is deliberately excluded so the built-in 低估值 demo seed keeps its existing rank(close) path.
  • both parse modes now assemble warnings through one parse_warnings(formula, text) helper (dedup, order-preserving), so an out-of-scope idea carries a review warning even when the formula is a valid non-fallback.
  • no data change, no prompt change, no field expansion, no formula change — the formula for such an idea is still the price-volume approximation; it just no longer masquerades as a faithful implementation. The web payload already surfaces parsed.warnings, so the frontend warning card shows it with no UI change.

Tests

  • reproduces the exact reported case (fundamentals idea → rank(return_5d) + review warning, and asserts it is NOT the rank(close) fallback warning);
  • recall + no-false-positive sweep (fundamentals/valuation/alt-data warn; the demo seeds and price-volume ideas don't; short English terms don't match inside approach/broad);
  • verified end-to-end against real DeepSeek. Affected suites green (182), release_safety_scan.py passes.

Quant Forge contributors added 2 commits July 17, 2026 01:50
full_integration_test_prompt.md:
- new 4.1.1 acceptance sub-section for the Web-UI runtime LLM switch
  (POST /api/settings/llm): preset visibility, register+inject key, key
  secrecy, the base_url-redirect guard, switch-back no-regression, token gating;
- section 5 gains a backend cross-check for /api/settings/llm;
- the read-only surfaces + IA tab list use the renamed 已注册因子 label
  (internal ids lab-tab-registry/lab-panel-registry unchanged).

integration_workflow.md: the 4.2 config-input checklist gains a 4b step
for the runtime provider switch (preset register, base_url guard, switch-back)
and notes the preset entries in the provider dropdown.
The public workbench prompt only allows a handful of price-volume formulas
and the panel has no fundamental/valuation/alt-data fields, so a
fundamentals idea (e.g. 年报季报利润同比) silently degraded to a
confident-looking rank(return_5d) with no warning -- the no-silent-fallback
contract only tripped on the rank(close) catch-all.

Add out_of_scope_data_warnings(text) in specs/nl_flow (single source of the
honesty contract): a curated, conservative keyword detector for
fundamentals / valuation-multiple / alt-data domains the panel can't
express. Both parse modes now assemble warnings through a single
parse_warnings(formula, text) helper (dedup, order-preserving), so an
out-of-scope idea carries an explicit review warning even when the formula
is a valid non-fallback. The bare 估值 term is intentionally excluded so the
built-in 低估值 demo seed keeps its existing rank(close) path. The web parse
payload already surfaces parsed.warnings, so the frontend warning card shows
it with no UI change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant