Skip to content

Commit 660f50c

Browse files
committed
Preserve catalog gates through init reruns and parallel deepening
1 parent 91510ae commit 660f50c

13 files changed

Lines changed: 1128 additions & 31 deletions

File tree

‎doc/architecture/SPEC/kb_service.md‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -596,11 +596,15 @@ proof `basis` 缺省 supported,旧区块不改写。verified_absent 当前只
596596
判定时的编号源码正文须完整覆盖每个区块证明并匹配内容哈希,缺失区块还须绑定相同的
597597
重放证书;显式报告非 Codex 裁判身份则拒绝,未报告身份仍保留未知。
598598

599-
`--unlimited-subscription` 仅用于 knowledge-deepen/deepen,显式要求生成和裁判两角色
599+
`--unlimited-subscription` 用于 feature-discovery、modules、knowledge、knowledge-deepen
600+
及 widen/deepen 别名,显式要求生成和裁判两角色
600601
分别使用 Zcode GLM-5.3 与独立 Codex 的已认证订阅,拒绝其他模型协议、budget_usd 冲突
601602
及 configured fallback。预算上限 None 表示
602603
该模式,不使用无穷或零价格伪装;累计固定记账和原生调用用量仍记录,实际费用未报告
603604
保持未知。每次调用重新绑定已经检查的订阅传输,不启用 API 付费回退。
605+
modules/knowledge 的公共阶段预算仅在显式启用时改为 None;单独选择订阅生成器
606+
仍保留原记账上限。该模式绑定检查点身份,无此选项时的旧摘要保持不变;
607+
feature-discovery 与 knowledge-deepen 继续使用各自已有的检查点及身份实现。
604608

605609
### 2026-10-02 轻量认可、13 功能任务与轨迹留存
606610

@@ -628,6 +632,12 @@ Zcode 调度文件。默认启动间隔 15 秒、原生 429/account-1302 退避
628632
调度等待、开始、冷却与停止事件与原生输入输出一起脱敏保存于每个 worker 的
629633
`init/traces/attempts/`,最终关联唯一 model_call;未完成调用及未报告账单费用保持未知。
630634
监控读取错误记录诊断、沿用最后安全快照,不停止其它任务。
635+
发现目录合并后的并行深度批次使用 `--discovery-record` 显式传入原始完成记录。
636+
父任务复用正式阶段的前置门禁,检查真实 PR 已合并或原记录为 empty、done、repo/pin,
637+
以及基线中的目录与报告哈希;原 JSON 字节摘要进入批次身份,逐 worker 原样复制,
638+
不重写状态或外部原生档案路径。基线已有发现报告时,即使旧适配器未启用必需标记,
639+
也不能省略交接。worker 在模型初始化前再核对父任务绑定,记录变化或既有副本冲突
640+
拒绝恢复;没有发现报告的旧流程保持原行为。
631641
新增轻量区块的原生审计还须绑定实际 GLM-5.3 草稿的对应维度、正文和已展示引用,以及
632642
Zcode 身份和无 fallback 记录;附近的无关生成调用不能证明该区块的提取归属。
633643

@@ -867,3 +877,12 @@ Zcode 模型角色中的 `effort` 被 transport 忽略,不能代替该设置
867877
只能称“声明库存已处理”,不能证明所有功能均已发现。新目录的深度分母为 `N × 7`;
868878
历史目录的认可率独立保留,不能直接混用。功能发现、知识认可、生产文件结构覆盖、
869879
测试覆盖和真实 PR 评审效果是不同指标。
880+
881+
首次评审支持的生产源码新候选,进入目录前必须再经一次批量 Codex 目录边界审计。
882+
该审计展示完整的正式功能 ID、标题和 owner,不让正式功能与候选竞争 16000 字符预算;
883+
有剩余空间才展示相关候选。保留首次评审,另存真实原生审计回执、审计版本、种子哈希、
884+
候选/首次评审集合哈希及每项候选哈希。拒绝、通道失败、别名循环或未解决的重复标题/别名
885+
保持未知,不触发另一轮提取修正。跨候选别名只可归属最终已认可的正式或新增父功能。
886+
未发布的旧检查点缺少此审计时只补审,复用完整扫描和首次评审;prepared 产物缺少审计时
887+
阻止发布,保留原产物。已发布历史目录保持不可变。审计预算中断保存已完成批次,恢复时
888+
不重复已完成的审计。显式 `--retry-unfinished` 不重置已用完的三次内容修正,仅恢复通道故障。

‎eval/feature-discovery/README.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,12 @@ infermatrix-copilot kb init REPO --stage feature-discovery --from-existing \
3232
字段时使用新状态目录,保留旧轨迹。扩大包不会证明已发现所有功能,也不取消独立评审。
3333
恰好达到每包 24 个候选上限的任务另列数量和 ID,保留为潜在遗漏线索。
3434

35+
新增正式功能在首次认可后,还须经过一次目录边界审计。独立 Codex 同时看到所有既有
36+
功能的 ID、标题和 owner,并读取候选的真实源码引用;子能力、别名和实现补充归属父功能。
37+
首次评审及轨迹保留,边界审计另绑定版本、种子和候选集合哈希。重复标题/别名仍无法归属
38+
时保持未知,不增加分母。未发布旧批次可以复用扫描与首次评审补做这一步;内容修正已经
39+
达到三次上限的候选不会因显式重试再获得三次机会。
40+
3541
检查点绑定源码、范围、种子目录、索引版本、提示词与模型配置。成功分片、评审和待审修正稿可复用;预算中断不标为完成。竞争启动同一批次不会覆盖检查点。单项最多三次修正;通道故障保持未知,不重做已成功的提取。仅有文档的未知声明不通过重复改写变成实现证明。
3642

3743
紧凑摘要写入 `eval/feature-discovery/`。完整输入、原生输出、工具事件、配置、用量和修正过程留在 Git 外。恢复时核验每个完成分片的原生归档,包括返回空候选的分片。Zcode 保存原生流式事件;Codex 保存 CLI 结束后按原顺序转发的事件。

‎eval/knowledge-depth/README.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -73,6 +73,8 @@ JiuwenSwarm explicitly selects `semantic_depth.acceptance_mode: lightweight`; ot
7373

7474
The campaign runner defaults to 13 isolated workers and accepts `--workers 13`. Each worker owns its checkpoint, model gateway and trace archive. The parent builds one immutable source/test/document index bound to the source pin, policy and index version. Initial extraction packets offer up to 24 KB of source/test lines, 8 KB of relevant document lines and 4 KB of existing knowledge; subsequent corrections retrieve evidence for the specific gap. Format, references, hashes, receipts and counts are checked incrementally; comprehensive audits, retrieval acceptance and full CI run at delivery.
7575

76+
For a catalog produced by feature discovery, pass `--discovery-record /absolute/path/to/init/<repo>/feature-discovery.json` to `run_depth_campaign.py`. Use the original completed `published` record after its PR merges, or the original `empty` record when the catalog was already current. The native prerequisite gate checks the real PR state, source pin and exact catalog/report hashes at the campaign baseline. The campaign binds the record's byte hash and copies its exact bytes into every worker before dispatch; native trace paths stay unchanged outside Git. A baseline containing a discovery report requires this explicit handoff, including repositories whose older manifest did not require discovery. Changed records or mismatched worker prerequisites require a fresh campaign; statuses and prerequisite records are never synthesized. Legacy baselines without discovery retain their existing behavior.
77+
7678
Each missing facet receives an initial attempt and at most three corrections. The round is saved before dispatch, including review-only retries. Restarting or `--retry-unfinished` does not reset this lifetime limit. Passed facets are frozen; failure in another facet does not discard them. After exhausted repairs, the facet remains unknown with its reason. Creating the campaign's STOP file stops new dispatch and drains active calls; the supervisor records progress in `drain.json`.
7779

7880
Validation knowledge labels runtime assertions, source-text assertions, helper-unit assertions or existing documented manual checks. Manual entries include their documented action and expected observation and explicitly say this batch did not execute them. Missing tests never become verified absence, and knowledge about assertions does not mean tests passed.

‎eval/knowledge-depth/run_depth_campaign.py‎

Lines changed: 131 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,109 @@ def git(*args):
2525
subprocess.run(["git", *map(str, args)], check=True, stdout=subprocess.DEVNULL)
2626

2727

28+
def baseline_inputs(args):
29+
"""Resolve the adapter from the immutable baseline, including aliased dirs."""
30+
import yaml
31+
from infermatrix_copilot.kb_service.knowledge_coverage import policy_path
32+
33+
names = subprocess.check_output(["git", "-C", str(args.root), "ls-tree", "-r", "--name-only",
34+
args.baseline, "adapters"], text=True).splitlines()
35+
found = []
36+
for name in names:
37+
if not re.fullmatch(r"adapters/[A-Za-z0-9_-]+/manifest\.yaml", name):
38+
continue
39+
text = subprocess.check_output(["git", "-C", str(args.root), "show", args.baseline + ":" + name], text=True)
40+
manifest = yaml.safe_load(text)
41+
if isinstance(manifest, dict) and isinstance(manifest.get("knowledge"), dict) \
42+
and manifest["knowledge"].get("repo_subdir") == "repos/" + args.repo:
43+
found.append((name, manifest))
44+
if len(found) != 1:
45+
raise ValueError("campaign baseline must contain exactly one adapter for this knowledge repository")
46+
name, manifest = found[0]
47+
relative = policy_path(args.repo, Path(name).parent)
48+
text = subprocess.check_output(["git", "-C", str(args.root), "show", args.baseline + ":" + relative], text=True)
49+
return manifest, text, relative
50+
51+
52+
def discovery_handoff(args):
53+
"""Validate a genuine completed record against the merged frozen catalog."""
54+
from types import SimpleNamespace
55+
from infermatrix_copilot.kb_service.init_stages import _Chain, _Stage
56+
from infermatrix_copilot.kb_service.init_support import InitRecord
57+
from infermatrix_copilot.kb_service.sources import KnowledgeRepo
58+
59+
manifest, _, relative = baseline_inputs(args)
60+
report_path = f"eval/feature-discovery/{args.repo}-{args.pin[:12]}.json"
61+
knowledge = KnowledgeRepo(args.root)
62+
report = knowledge.show(args.baseline, report_path)
63+
lifecycle = manifest.get("knowledge_lifecycle") or {}
64+
configured = lifecycle.get("init") or {} if isinstance(lifecycle, dict) else None
65+
if not isinstance(configured, dict):
66+
raise ValueError("campaign baseline lifecycle/init must be a mapping")
67+
path = getattr(args, "discovery_record", None)
68+
if path is None:
69+
if report is not None or configured.get("feature_discovery_required"):
70+
raise ValueError("this campaign baseline requires --discovery-record; the frozen catalog gate cannot be omitted")
71+
return None
72+
path = Path(path).resolve()
73+
raw = path.read_bytes()
74+
record_hash = hashlib.sha256(raw).hexdigest()
75+
expected = getattr(args, "discovery_record_sha256", None)
76+
if expected is not None and record_hash != expected:
77+
raise ValueError("discovery prerequisite record changed after campaign binding")
78+
data = json.loads(raw)
79+
if not isinstance(data, dict) or not isinstance(data.get("discovery"), dict):
80+
raise ValueError("discovery handoff record must contain a discovery mapping")
81+
record = InitRecord(**data)
82+
if record.stage != "feature-discovery" or record.repo != args.repo or record.pin != args.pin \
83+
or record.status not in ("published", "empty") or record.discovery.get("done") is not True \
84+
or record.status == "published" and record.dry_run:
85+
raise ValueError("discovery handoff requires the original published/empty completed record for this repository and pin")
86+
# Reuse the production gate: verify real PR state, complete report, feature
87+
# IDs, owners and the exact catalog/report hashes at the merged baseline.
88+
# The temporary file is the original record's bytes, never a replacement
89+
# status or a fabricated prerequisite. Native archives stay at their origin.
90+
with tempfile.TemporaryDirectory(prefix="discovery-handoff-") as scratch:
91+
state = Path(scratch)
92+
copied = InitRecord.path(state, args.repo, "feature-discovery")
93+
copied.parent.mkdir(parents=True)
94+
copied.write_bytes(raw)
95+
stage = _Stage(SimpleNamespace(state_dir=state, knowledge=knowledge, gh_run=subprocess.run),
96+
SimpleNamespace(repo=args.repo, knowledge_dir="repos/" + args.repo,
97+
adapter_dir=Path(relative).parent,
98+
init=SimpleNamespace(feature_discovery_required=True)),
99+
dry_run=False, pin=args.pin)
100+
stage.STAGE = "knowledge-deepen"
101+
stage.repo_dir, stage._base_sha = "repos/" + args.repo, args.baseline
102+
chain = _Chain()
103+
stage._discovery_gate(chain, args.pin)
104+
if chain.problems:
105+
raise ValueError("discovery handoff failed the merged catalog gate: " + "; ".join(chain.problems))
106+
binding = stage._discovery_binding()
107+
return {"record_path": str(path), "record_sha256": record_hash,
108+
"report_path": report_path, **binding}
109+
110+
111+
def install_discovery_handoff(args, state, binding):
112+
"""Copy the exact immutable prerequisite into a worker without rewriting it."""
113+
if binding is None:
114+
return
115+
from infermatrix_copilot.kb_service.init_support import InitRecord
116+
117+
raw = Path(binding["record_path"]).read_bytes()
118+
if hashlib.sha256(raw).hexdigest() != binding["record_sha256"]:
119+
raise ValueError("discovery prerequisite record changed before worker handoff")
120+
dest = InitRecord.path(state, args.repo, "feature-discovery")
121+
if dest.exists():
122+
if dest.read_bytes() != raw:
123+
raise ValueError("worker discovery prerequisite differs from the immutable campaign record")
124+
return
125+
dest.parent.mkdir(parents=True, exist_ok=True)
126+
temporary = dest.with_suffix(".json.tmp")
127+
temporary.write_bytes(raw)
128+
os.replace(temporary, dest)
129+
130+
28131
def worker(args):
29132
state = args.state / f"worker-{args.worker}"
30133
state.mkdir(parents=True, exist_ok=True)
@@ -53,6 +156,12 @@ def run_worker(args, state):
53156
previous = InitRecord.load(state, args.repo, "knowledge-deepen")
54157
if previous and (previous.kb_base_sha != args.baseline or previous.pin != args.pin):
55158
raise RuntimeError("worker checkpoint source or knowledge baseline differs from campaign")
159+
binding = discovery_handoff(args)
160+
if binding is not None:
161+
metadata = json.loads((args.state / "campaign.json").read_bytes())
162+
if metadata.get("discovery") != binding:
163+
raise RuntimeError("worker discovery handoff differs from the frozen campaign identity")
164+
install_discovery_handoff(args, state, binding)
56165
runtime = InitRuntime.from_env(Settings(_env_file=None), state_dir=state)
57166
from infermatrix_copilot.kb_service.depth_pacing import SharedZcodePacer
58167
runtime.gateway.configure_zcode_pacing(SharedZcodePacer(
@@ -88,6 +197,9 @@ def main():
88197
parser.add_argument("--depth-index-path", type=Path)
89198
parser.add_argument("--repair-guidance", type=Path, help="Pinned, immutable localization hints for a newly authorized gap-repair batch")
90199
parser.add_argument("--repair-guidance-sha256", help=argparse.SUPPRESS)
200+
parser.add_argument("--discovery-record", type=Path,
201+
help="Original completed, merged feature-discovery record; required for a discovered catalog")
202+
parser.add_argument("--discovery-record-sha256", help=argparse.SUPPRESS)
91203
parser.add_argument("--zcode-pacing-path", type=Path)
92204
parser.add_argument("--zcode-start-interval", type=float, default=15.0)
93205
parser.add_argument("--zcode-rate-cooldown", type=float, default=90.0)
@@ -105,6 +217,8 @@ def main():
105217
args.depth_index_path = args.depth_index_path.resolve()
106218
if args.repair_guidance:
107219
args.repair_guidance = args.repair_guidance.resolve()
220+
if args.discovery_record:
221+
args.discovery_record = args.discovery_record.resolve()
108222
if args.zcode_pacing_path:
109223
args.zcode_pacing_path = args.zcode_pacing_path.resolve()
110224
from infermatrix_copilot.kb_service.depth_pacing import SharedZcodePacer
@@ -118,6 +232,8 @@ def main():
118232
parser.error("worker must belong to the campaign and have feature IDs")
119233
if args.repair_guidance and not re.fullmatch(r"[0-9a-f]{64}", args.repair_guidance_sha256 or ""):
120234
parser.error("guided workers require their parent's immutable guidance hash")
235+
if args.discovery_record and not re.fullmatch(r"[0-9a-f]{64}", args.discovery_record_sha256 or ""):
236+
parser.error("discovery workers require their parent's immutable prerequisite hash")
121237
return worker(args)
122238
args.baseline = subprocess.check_output(["git", "-C", str(args.root), "rev-parse", args.baseline + "^{commit}"], text=True).strip()
123239
args.state.mkdir(parents=True, exist_ok=True)
@@ -130,13 +246,11 @@ def main():
130246

131247

132248
def campaign(args, parser):
133-
from infermatrix_copilot.kb_service.knowledge_coverage import load_policy, policy_path
249+
from infermatrix_copilot.kb_service.knowledge_coverage import load_policy
134250
from infermatrix_copilot.kb_service.sources import KnowledgeRepo
135-
import yaml
136251
from infermatrix_copilot.kb_service.outbox import atomic_write_json
137252

138-
manifest = yaml.safe_load((args.root / f"adapters/{args.repo}/manifest.yaml").read_text())
139-
text = subprocess.check_output(["git", "-C", str(args.root), "show", args.baseline + ":" + policy_path(args.repo)], text=True)
253+
manifest, text, _ = baseline_inputs(args)
140254
policy = load_policy(text, manifest["knowledge"]["repo_subdir"])
141255
if policy.semantic_depth_per_facet_gt is None:
142256
parser.error("campaign baseline must declare its global semantic completion target")
@@ -156,6 +270,12 @@ def campaign(args, parser):
156270
"features": len(policy.features), "denominator": len(policy.features) * 7,
157271
"workers": args.workers, "partitions": partitions,
158272
"acceptance_mode": getattr(args, "acceptance_mode", "strict")}
273+
try:
274+
binding = discovery_handoff(args)
275+
except (ValueError, TypeError, OSError) as exc:
276+
parser.error(str(exc))
277+
if binding is not None:
278+
summary["discovery"] = binding
159279
if getattr(args, "repair_guidance", None):
160280
from infermatrix_copilot.kb_service.depth_inputs import load_repair_guidance
161281
guidance = load_repair_guidance(args.repair_guidance, pin=args.pin,
@@ -206,6 +326,10 @@ def campaign(args, parser):
206326
jobs, checkpoints = [], []
207327
for number in range(args.workers):
208328
state = args.state / f"worker-{number}"
329+
try:
330+
install_discovery_handoff(args, state, binding)
331+
except (ValueError, OSError) as exc:
332+
parser.error(str(exc))
209333
clone = state / "init/knowledge-repo"
210334
if not clone.exists():
211335
clone.parent.mkdir(parents=True, exist_ok=True)
@@ -227,6 +351,9 @@ def campaign(args, parser):
227351
if getattr(args, "repair_guidance", None):
228352
argv += ["--repair-guidance", str(args.repair_guidance),
229353
"--repair-guidance-sha256", summary["repair_guidance_sha256"]]
354+
if binding is not None:
355+
argv += ["--discovery-record", binding["record_path"],
356+
"--discovery-record-sha256", binding["record_sha256"]]
230357
if args.retry:
231358
argv.append("--retry")
232359
jobs.append((number, argv))

0 commit comments

Comments
 (0)