[napetrov]

Part of #4.
Depends on #164 and #168: rebase after both merge (routing.py and the routing tests overlap with those branches).

Summary

novelty routing builds its field the way a harness does: the catalog's own manifest, without skills the model cannot select, and with each mirror counted once. A new --all-vs-all mode routes every skill's queries against the whole field and reports who takes whose queries.

 cmd_novelty_routing
+  --all-vs-all --queries-by-skill <file> --out <file>
+    every skill with queries is the gold in turn
+    confusion matrix, own_rate per skill, attractors (skill picked for other skills' queries)
+    (none) = no card shares a word with the query
   field = _routing_field(args)
-    --catalog: every SKILL.md under root
+    --catalog: skills.yaml when present, else every SKILL.md
+    skip disable-model-invocation        (--include-disabled keeps)
+    skip non-SKILL.md files and dirs without SKILL.md, with a warning
+    one copy per byte-identical SKILL.md (nearest the catalog root)
+    same name, different bytes -> refuse, naming both paths

Evidence

Eleven new tests fail against intel/main; the other 68 in the file pass there. The run reverts commands/novelty.py only, because the new tests import all_vs_all, load_queries_by_skill and disables_model_invocation from novelty/routing.py, so reverting that file too would fail the whole module on import:

before: commands/novelty.py reverted to intel/main, rest of this branch kept
  FAILED test_a_non_skill_file_among_competitors_is_skipped_with_a_warning
  FAILED test_a_disable_model_invocation_skill_is_not_a_competitor_unless_asked
  FAILED test_an_explicit_disabled_competitor_is_skipped_with_a_warning
  FAILED test_a_disabled_artifact_is_still_routed_but_warned_about
  FAILED test_a_mirror_of_a_catalog_skill_is_counted_once
  FAILED test_two_catalog_skills_with_one_name_and_different_content_are_refused_by_name
  FAILED test_all_vs_all_writes_the_matrix_and_prints_the_attractors
  FAILED test_all_vs_all_with_model_names_its_model_calls_on_stderr
  FAILED test_all_vs_all_without_model_says_nothing_about_model_calls
  FAILED test_all_vs_all_takes_no_artifact
  FAILED test_a_single_routing_run_still_needs_an_artifact
  11 failed, 68 passed

The routing.py-level tests (the attractor table, the (none) rule, the disable-flag spelling in test_the_disable_flag_is_read_the_way_the_repo_budget_reads_it) need the new module and cannot be run against intel/main at all.

Local checks on the rebased head:

uv run ruff check .                              All checks passed!
uv run mypy dokimi                               Success: no issues found in 222 source files
uv run python -m dokimi.schema_check             exit 0
uv run python -m dokimi.config_check             exit 0
uv run pytest -n auto -q                         7099 passed, 22 skipped

Merge Danger

Door: two-way. Every mode is additive except the default field, which now differs for catalogs with disabled or mirrored skills. Reverting restores the old field.

Blast Radius: medium. Existing routing figures change for any catalog that has disabled or duplicated skills, so older numbers are not comparable with new ones.

• Default excludes disable-model-invocation skills. A --catalog or --competitors field no longer includes a skill that sets disable-model-invocation, and an artifact with that flag is routed with a warning. To keep the old field, pass --include-disabled.
• Catalog reads the manifest. With skills.yaml present, --catalog takes its entries, and a declared skill that does not load is skipped with a note rather than aborting the run.
• Mirrors counted once. A byte-identical SKILL.md under another path is dropped, keeping the copy nearest the catalog root.
• Name clash refuses. Two catalog skills with one name and different bytes now raise, naming both paths.
• Non-skill competitors skipped. A skill: reference to a file that is not SKILL.md, or to a directory without one, is skipped with a warning. text: and plain paths are unchanged.
• New mode. --all-vs-all makes one selector call per query that has a content word (a query without one is unscored in both arms and not sent) over the whole field when --model is given. The CLI prints the call count on stderr before it starts.
• --artifact is now optional only with --all-vs-all; a single run without it still fails.
• No persisted-format change beyond the new all_vs_all payload written to its own --out file.

Checklist

• Tests added/updated and pytest passes
• ruff, mypy, and schema checks pass
• Docs updated: docs/skill-novelty.md and docs/skill-novelty-walkthrough.md
• No ad-hoc run artifacts committed
• New code and test files carry the SPDX license header (no new code files; changelog fragments follow the existing fragments, which carry none)

🤖 Generated with Claude Code

[antonwolfy]

Review at 3ab7f1b. The field logic holds up: the manifest read, the name-clash check before the mirror dedupe, and the disable-flag spellings behave as described. The problems are in the new --all-vs-all mode and its tests. Five things are worth fixing before merge, and one nit.

What I ran, on a Linux review host at 3ab7f1b:

• make ci-local: 20 passed, 8 failed, 7 skipped. The 8 failures are exit 127 (ruff, zizmor, pre-commit, bandit, mypy, deptry, mkdocs, pip-audit missing from my venv): make dev stops at the tools install on this base, which #179 fixed on main. Static checks, mypy and the strict docs build pass on GitHub's CI at this head. The test job passed: 7,100 passed, 21 skipped, changed-line coverage gate passed.
• tests/test_novelty_routing.py: 79 passed.
• Reproductions for the code findings, with the LLM transport mocked.
• A mutation run of about 60 mutants on the new code: 35 killed, the survivors are in the test comment.

Checked, no issues

• Disable flag: true, True, 'true', yes, on, 1 read as disabled; false, no, y do not. This matches repos/budget.py.
• Clash and mirrors: a disabled skill that shares a name with an enabled one is dropped first, so there is no false clash. Ordering is deterministic.
• Call count: the stderr count equals the calls made on a valid queries file.
• Confusion and attractor figures: own_rate, wrongly_picked, sibling_queries match hand counts, and a query with no shared word is (none).
• Changelog fragments: format and location match the existing ones.

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 91.35802% with 28 lines in your changes missing coverage. Please review.
✅ Project coverage is 92.80%. Comparing base (8889feb) to head (9490195).
⚠️ Report is 6 commits behind head on main.

Files with missing lines	Patch %	Lines
dokimi/dokimi/commands/novelty.py	86.36%	10 Missing and 8 partials ⚠️
dokimi/dokimi/novelty/routing.py	89.70%	2 Missing and 5 partials ⚠️
dokimi/dokimi/external/catalog.py	60.00%	1 Missing and 1 partial ⚠️
dokimi/dokimi/novelty/all_vs_all.py	99.15%	1 Missing ⚠️
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #172      +/-   ##
==========================================
+ Coverage   92.76%   92.80%   +0.04%     
==========================================
  Files         261      262       +1     
  Lines       32898    33280     +382     
  Branches     5503     5578      +75     
==========================================
+ Hits        30518    30886     +368     
- Misses       1581     1584       +3     
- Partials      799      810      +11     

Files with missing lines	Coverage Δ
dokimi/dokimi/novelty/all_vs_all.py	99.15% <99.15%> (ø)
dokimi/dokimi/external/catalog.py	96.57% <60.00%> (-0.62%)	⬇️
dokimi/dokimi/novelty/routing.py	96.60% <89.70%> (-2.41%)	⬇️
dokimi/dokimi/commands/novelty.py	94.75% <86.36%> (-1.60%)	⬇️

... and 5 files with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[antonwolfy]

I pushed a commit per nit, up to 70b7a72, which fix the nit from my review:

• 70b7a72: dokimi/docs/skill-novelty.md:602: the sentence now says a disabled artifact is still routed, with a warning, and the flag does not change that; the doubled "before quoting either:" at line 647 is dropped.

make ci-local at 70b7a72: 20 passed, 8 failed, 7 skipped.

Not approving yet. Still open: the bug and the four risks from my review.

[antonwolfy]

Review at 924aba9. All five points from my earlier review are fixed and tested, and the field logic still holds. Two things are worth fixing before merge, both in the new --all-vs-all command, and there are two nits inline plus one in the body.

What I ran, on a Linux review host at 924aba9:

• make ci-local: 28 passed, 1 failed, 7 skipped. The failure is pip-audit on gitpython 3.2.0 (CVE-2026-114145, fix 3.2.1), a pinned dependency the PR does not touch. Tests: 7,258 passed, 1 skipped. The changed-line coverage gate passed.
• tests/test_novelty_routing.py and tests/test_novelty_routing_all_vs_all.py: 99 passed.
• Reproductions for the code findings, with the LLM transport mocked.
• A mutation run of about 55 mutants on the new code. The survivors from my earlier review are killed, except gold=0, which you explained. Several new ones survive (below).

Nit: the all-vs-all tests leave a few outputs unpinned. With the full suite at 99 passed, these mutants survive: seed=0 passed to all_vs_all from _cmd_routing_all_vs_all; the top-five cap in _print_attractors changed to three; the name tie-break on equal wrongly_picked (and in from) dropped or reversed; the gold != name filter removed from sibling_queries, which only changes rate for an attractor that has queries of its own (the exact-row tests use a hub with none). Suggestion: add one case with two attractors tied on count and one attractor that also has queries, and assert seed in the saved payload.

Checked, no issues

• Tie scoring: two identical cards give confusion {"a": {"b": 1}, "b": {"a": 1}} and own_rate 0.0 in both field orders. A query with no shared word is (none).
• Failed calls: a transport that raises on every call gives n_failed equal to the query count, a stderr warning and a non-zero exit with the partial result written.
• Queries file: a missing file, a non-UTF-8 file, a top-level list, null, non-string items, an empty file and an unknown skill name each exit with a message, not a traceback.
• --max-usd: with a cap of 1.0 and a mock priced at 0.6 per call, the run stops after 2 calls with spend cap reached, and no --out file is written. The docs now say so.
• Field logic: true, True, 'true', yes, on, 1 read as disabled; false and 'off' do not. Byte-identical mirrors collapse to the copy nearest the root, and a same-name clash with different bytes names both paths.
• Docs and changelog: the flags, outputs and defaults in skill-novelty.md, the walkthrough and the five fragments match the code.

[xaleryb]

• Checks run: the full suite, ruff, mypy and bandit on the head merged with current main; every point below was reproduced on a toy catalog.
• --all-vs-all own_rate counts unscored queries as misses. alpha queries ["convert my json file to yaml","json to yaml please","the of and"] -> single mode R@1 1.0 over 2 of 3; all-vs-all own_rate 0.6667. A skill whose model calls all failed reads 0.0, the same as one the model really never picked. Expected: the single-mode denominator (scored/attempted queries), plus per-skill counts.
• A lexical tie between two or more non-gold cards goes to the first card in the field. Three identical cards: order [fft, linalg, random] -> top attractor fft; reversed order -> random. Expected: the same result in any order, e.g. a (tie) bucket.
• routing.json does not record the field rule. A default run and an --include-disabled run write the same keys. Expected: include_disabled, the competitor names and the left-out names in both payloads.
• Pushed: d992d46 (the model arm records model); adbe542 (test that pins the attractor rate for a skill that has queries of its own; the old guard never ran).
• Left for the author: the three points above.

[antonwolfy]

Review at adbe542. The two commits since 924aba9 are fine: model is recorded in the model arm and the new attractor-rate test fails when the gold != name filter is removed. The four threads from my last review are unchanged at this head, so the PR is not ready: T1 and T2 are still risks.

What I ran, on a Linux review host at adbe542:

• make ci-local: 28 passed, 1 failed, 7 skipped. The failure is pip-audit, which fails on the lock files; this PR does not touch them. Tests: 7,261 passed, 1 skipped.
• Reproductions for the open threads, with the LLM transport mocked.
• A mutation check of the two new tests: removing model=client.model fails both model-a/model-b cases (2 failed, 31 passed); removing the gold != name filter in sibling_queries fails the new attractor test (1 failed, 32 passed).

Still open from my earlier review:

• dokimi/dokimi/commands/novelty.py:364: a partial model failure still exits 0 and prints OK Saved. Re-run at this head: 5 queries over db and perf, a transport that answers once then raises rate limited, gives exit 0, own_rate {db: 0.5, perf: 0.0} and n_failed 4.
• dokimi/dokimi/novelty/routing.py:541: {"a":["x"],"a":["y"]} still loads as {'a': ['y']} with no error.
• dokimi/dokimi/novelty/routing.py:600: the "not in the field" error still does not name disable-model-invocation or --include-disabled.
• dokimi/dokimi/commands/novelty.py:375: --queries and --queries-from-headings are still ignored under --all-vs-all when --queries-by-skill is also given; --queries /nonexistent.json --queries-by-skill q.json --all-vs-all --out o.json prints OK Saved all-vs-all routing.

@xaleryb's points on the own_rate denominator, the lexical tie among non-gold cards and the missing field rule in the payload wait for the author.

Checked, no issues

• Docs and changelog: the flags, outputs and defaults in skill-novelty.md, the walkthrough and the five fragments match the code, including the new model sentence; mkdocs build --strict passes.
• Repo rules: SPDX header on the new test file, no schema or with_docs changes, no generated artifacts, changelog fragments in changelog.d/.

[napetrov]

Review-body items from @xaleryb's comment, in 5e27afa: (1) own_rate denominator: rates are now over scored queries; a query with no content word or a failed model call is unscored, reported in n_scored/n_unscored per skill, and a skill with none scored has own_rate null (test_a_query_with_no_content_word_is_not_scored_as_a_miss). (2) lexical tie among non-gold cards: two or more other cards tied at the top are a (tie) bucket in every field order, never an attractor (test_a_tie_among_other_cards_is_a_tie_in_any_field_order). (3) field rule: routing.json, single and all-vs-all, now carries field {include_disabled, competitors, left_out} (test_the_saved_payload_records_the_field_rule, test_a_single_artifact_routing_run_records_the_field_rule_too).

[github-actions]

CLI interface change detected

This PR changes dokimi's user-facing CLI surface (commands and/or options):

~ `dokimi novelty routing` parameters changed:
    + added option `--all-vs-all`
    + added option `--include-disabled`
    + added option `--queries-by-skill`
    ~ changed option `--artifact`: required: True -> False

Updated on every push; the cli-interface-change label is on while the diff is non-empty.

[antonwolfy]

Review at 898e053. The four threads from my last review are fixed and tested, and the field logic mostly holds. Five points are new: one bug in the mirror dedupe, three risks and one nit.

What I ran, on a Linux review host at 898e053:

• make ci-local: 28 passed, 1 failed, 7 skipped. The failure is pip-audit (filelock, setuptools), which fails on the lock files; this PR does not touch them. Tests: 7,827 passed, 1 skipped. The changed-line coverage gate passed.
• Reproductions for the code findings, with the LLM transport mocked.
• A mutation run of 13 mutants on the 5e27afa behaviours in test_novelty_routing_all_vs_all.py: 10 killed, 2 survive (point 1), 1 equivalent.

Finding that cannot be anchored inline:

1. Two tie rules are not pinned by a test. Changing len(others) == 1 to <= 2 in the tie rule leaves all 44 tests in test_novelty_routing_all_vs_all.py passing: it brings back the field-order dependence for a tie of exactly two other cards. test_a_tie_among_other_cards_is_a_tie_in_any_field_order uses three other cards, and test_a_three_way_tie_is_not_a_perfect_pick asserts only own_rate == 0.0. Changing the lexical unscored.get(name, 0) + 1 to = 1 also passes, because the lexical n_unscored is only tested with one unscored query. Add a case with exactly two tied other cards that asserts (tie) in the confusion row, in both field orders, and a case with two contentless queries for one skill that asserts n_unscored == 2.

Checked, no issues

• Earlier threads: a partial model failure writes the result and exits non-zero; a repeated key in the queries file is refused; the not-in-the-field error names --include-disabled; --queries and --queries-from-headings are refused under --all-vs-all.
• Disable flag: true, yes, TRUE and true # c read as disabled; false and an absent key do not. This matches repos/budget.py.
• Manifest and field record: an unlisted SKILL.md is not a sibling under skills.yaml, a ghost entry is a warning plus a left_out line, and the flags and payload keys in the docs match the code.
• Spend cap: SpendCapReached still passes through model_routing's except Exception.

[antonwolfy]

I pushed a commit per nit, up to b5830ba, which fix the nit from my review:

• b5830ba: dokimi/changelog.d/20261021_133000_napetrov_routing-mirror-dedupe.md:3: the entry now describes what is new, that --catalog with a skills.yaml counts byte-identical SKILL.md entries once and names the dropped copies on stderr.

make ci-local at b5830ba: 26 passed, 3 failed (host: gitleaks install and scan, pip-audit as at the reviewed head), 7 skipped.

GitHub CI has not run on these commits: the branch conflicts with main, and Actions runs no workflow on a conflicting PR. CI will check them once main is merged in.

Not approving yet. Still open: the bug and the three risks from my review.

[vchamarthi]

Not approving yet: 1 item to fix before merge.

One bug blocks merge: the mirror dedupe keys on file bytes alone and drops distinct skills that share text. The branch also conflicts with main, so no workflow has run at this head. The three open threads from the last review (console summary on ties and unscored queries, model arm scoring contentless queries) are not repeated here.

CI at send time: 0 failing, 0 pending.

[vchamarthi]

Should fix: _keepers_by_digest keys on the digest alone, so two differently named skills with identical text collapse into one; key it on (skill.name, digest) as discover_skills does.

discover_skills at skills/loader.py documents that without a frontmatter name the name is the directory's, so identical text in skills/a/ and skills/b/ is two skills, and it keys on (_skill_name(meta, file), sha256). _eligible_catalog_skills returns (skill, digest), _keepers_by_digest stores keeper[digest], and catalog_competitors then drops every other skill with that digest as identical to .... Result: the field shrinks from two cards to one, the saved field.left_out calls a different skill a mirror, and an all-vs-all run loses a sibling, which is the confusable pair the mode exists to find. The same applies to two differently named skills.yaml entries that point at identical bytes. _refuse_conflicting_names does not catch it because it only compares copies of one name. Same point as the open inline thread by antonwolfy; a test with two differently named, same-text skills should accompany the fix.

[napetrov]

Fixed in 4596c9f, the same change as the thread on antonwolfy's review: _keepers_by_digest is now _keepers_by_copy, keyed on (skill.name, digest). Tests: test_two_differently_named_skills_with_identical_text_both_stay_in_the_field and test_two_manifest_entries_with_identical_text_and_different_names_both_stay, both red before and green after.

[vchamarthi]

Question: growth_baseline.json on main now has novelty.py at 922 lines plus a cmd_novelty_routing complexity entry, so this line conflicts; after rebasing, does the 1029 figure still match the merged file, and does cmd_novelty_routing still sit under its complexity budget now that it dispatches to _cmd_routing_all_vs_all?

PR head novelty.py is 1029 lines. main growth_baseline.json has "dokimi/commands/novelty.py": {"lines": 922, ...} and "dokimi/commands/novelty.py::cmd_novelty_routing": {"complexity": 13, ...}. The PR's one-line edit replaces the 864 entry that main has since changed. mergeable_state is dirty, and only CodeFactor and DCO reported at this head.

[napetrov]

On the current head novelty.py is 1111 lines. The growth_baseline entry is that exact value with a rationale; 083ca41 raised it from 1088 for the console caveat and the blind-skill warning. cmd_novelty_routing keeps its recorded complexity of 13 (cap 10) under its existing rationale, and the gate measures 13 and passes. The new warning lives in its own helper, _warn_blind_skills, so _cmd_routing_all_vs_all is under the cap and needs no entry.

[napetrov]

Pushed 470cf73 on top of b5830ba (fast-forward, no force). intel/main (b2a9473) is merged in with sign-off; the conflicts were in catalog.py, growth_baseline.json and test_novelty_routing.py, resolved by keeping main's refactor and this PR's tests.

Items from the two reviews at b5830ba:

• Mirror dedupe keyed on bytes alone (vchamarthi, antonwolfy): fixed in 4596c9f, keyed on name and digest. Tests: test_two_differently_named_skills_with_identical_text_both_stay_in_the_field and test_two_manifest_entries_with_identical_text_and_different_names_both_stay.
• Console summary reads "no skill was picked" where the result is least clean (antonwolfy): fixed in 083ca41. Tests: test_console_says_how_many_tied_and_unscored_queries_are_not_counted_as_picks, test_console_warns_when_the_lexical_arm_cannot_score_any_query_of_a_skill, test_two_contentless_queries_of_one_skill_are_two_unscored_and_no_rate.
• Changelog fragment claims a fix the PR does not make (antonwolfy): fixed in b5830ba.
• Garbled sentences in the docs (antonwolfy): fixed in 70b7a72.
• Growth baseline question (vchamarthi): answered in its thread; novelty.py is 1111 on this head and the entry says so.

Still open:

• The model-arm decision (antonwolfy, thread 4371375820): left for the owner, not changed here.
• Question from a second review pass: two differently named skills with identical text are now two cards, so in all-vs-all each is the other's attractor (a to b, 100%). The existing test test_two_identical_cards_tie_and_neither_is_scored_a_win already asserts that the pair ties, and the mirror fix makes it visible. Should such twins be reported as one tied group instead? Not changed; needs the author's call.

Not ready for another look until the model-arm decision is made.

[antonwolfy]

The mirror dedupe and the console caveat are fixed, and CI is green. The model-arm decision is still open, and two new tests leave real regressions uncaught. Two risks and one nit are listed below.

What I ran, on a Linux review host at 470cf73:

• make ci-local: 29 passed, 0 failed, 7 skipped. Tests: 8,449 passed, 2 skipped. The changed-line coverage gate passed.
• Reproductions for the findings, with the LLM transport mocked.
• A mutation run of 62 mutants on the new code and its tests; the survivors I re-ran are inline.

Still open from my earlier review:

• dokimi/dokimi/novelty/routing.py (model arm): a query with no content word is still sent to the selector. Field alpha/beta, queries {"alpha": ["the of and", "convert json"]}, a selector that always picks card 1: lexical n_scored {alpha: 1}, n_unscored {alpha: 1}, own_rate 1.0; model n_scored {alpha: 2}, n_unscored {}, own_rate 0.5, and 2 calls made. The docs (skill-novelty.md, the n_scored sentence) and the payload note still say such a query "is not scored" without limiting that to the lexical arm. Either skip it in the model arm and count it in n_unscored, or say in both places that the model arm scores every query.

Checked, no issues

• Mirror dedupe: keyed on (name, digest); skills/a and skills/b with identical text stay two cards, and a same-name mirror collapses to the copy nearest the root.
• Console summary: the tied and unscored counts equal the (tie) total in confusion and the sum of n_unscored in the saved JSON; the blind-skill warning fires only when n_scored is 0.
• Docs and changelog: the flags, output keys and defaults match the code; the mirror fragment now claims only the skills.yaml path.
• Growth baseline: novelty.py is 1111 lines and the entry says 1111.

[antonwolfy]

I pushed a commit per nit, up to 995b153, which fix the nit from my review:

• 995b153: dokimi/dokimi/novelty/routing.py:499: the Raises text now says a broken SKILL.md is skipped with a note, and that an invalid skills.yaml is what raises.

Left as is:

• dokimi/dokimi/novelty/routing.py: the model-arm point is a risk left for the author, and the body has no Nit: or Non-blocking: item.

make ci-local at 995b153: 29 passed, 0 failed, 7 skipped.

Not approving yet. Still open: the two risks from my review; the earlier thread of mine above nit level.

[napetrov]

Pushed 1de5d71 on top of 995b153 (no force). intel/main (a2b787a) is merged in with sign-off in fc49df7; the only conflict was scripts/growth_baseline.json: main's version is taken, check_growth.py --update run, and the novelty.py entry set to the merged file's 1174 lines with a rationale.

Items from @antonwolfy's reviews at 898e053 and 470cf73:

• Model arm scored queries the lexical arm leaves out (thread on routing.py): fixed in 1de5d71. A query with no content word is now unscored in both arms and not sent to the selector; the call count on stderr and the failed-call figure count only calls sent.
• Tied/unscored console line not pinned (and/or, sum mutants): tests added in 1de5d71, exact lines for tied-only, unscored-only and unscored summed over skills.
• Same-name copies that differ late in the file: test added in 1de5d71.
• Tie rule len(others) == 1 vs <= 2 (898e053 body, point 1): test_exactly_two_other_cards_tied_at_the_top_are_a_tie (both orders) in 1de5d71. The lexical unscored + 1 mutant is killed by test_two_contentless_queries_of_one_skill_are_two_unscored_and_no_rate (083ca41).
• Broken-SKILL.md docstring nit: fixed in 995b153.

Still open for the author's call: two differently named skills with identical text are two cards, so in all-vs-all they tie ((tie)/each other's pick). Reporting such twins as one tied group is not changed here.

Local: make ci-local 28 of 29 steps pass; the test step's failures were AF_UNIX path too long and a timing assertion from the long tmp path on this host; those files pass with a short TMPDIR (708 passed). Routing tests: 246 passed.

[antonwolfy]

Review at 1de5d71. The model arm now skips a query with no content word, as the lexical arm does, and the stderr call count and the failed-call figure follow it. I found no wrong behaviour in the new code. One nit inline: a line of that change is not pinned by any test. @napetrov's question, whether two differently named skills with identical text should be one tied group in all-vs-all, is still open for the author.

What I ran, on Linux at 1de5d71:

• make ci-local: 28 passed, 0 failed, 7 skipped. Tests: 9,057 passed, 2 skipped. The changed-line coverage gate passed.
• Reproductions with the LLM transport mocked: a skill whose queries are all contentless, an abstaining selector, and contentless plus failed calls on one skill. The stderr call count equals the calls made, and the "N of M model call(s) failed" figure counts only calls sent.
• A mutation run of 15 mutants on the changed lines of _model_arm and _cmd_routing_all_vs_all: 13 killed, 1 equivalent (the early continue), 1 survives (inline).
• Mutants for my two test threads: and for or in the console line fails 3 of the new console tests, and hashing read_bytes()[:50] fails the new late-difference test.

Checked, no issues

• Call count: calls in commands/novelty.py filters with the same tokenize as _model_arm; dropping the filter or counting every query fails a test.
• Docs and changelog: the n_scored sentence, the cost paragraph, the payload note and the changelog fragment now say a contentless query is unscored in both arms and not sent.
• Growth baseline: novelty.py is 1174 lines and check_growth.py passes.

[xaleryb]

• Re-ran at e616ef1, merged with current main: the full suite, ruff, mypy, bandit, the growth check, the three earlier own_rate/tie/field-record inputs (all now pass), and every commit since then on toy catalogs with a mocked selector.
• --all-vs-all --model m --repeats 5 -> 2 calls for 2 queries, exit 0, and the payload has no repeats -> expected: refused, the same way --queries is refused.
• Pushed c8a9592: --all-vs-all refuses --repeats > 1. Includes a test that fails without the fix, and the docs and the all-vs-all fragment now say it.
• Left for the author: the existing question of whether two differently named skills with identical text should be reported as one tied group.

[antonwolfy]

LGTM. The two commits since 1de5d71 do what they say: --all-vs-all now refuses --repeats above 1 instead of dropping it, and the failed-call accumulation onto contentless queries is pinned by a test. No new findings.

What I ran, on Linux at c8a9592:

• make ci-local: 26 passed, 2 failed, 7 skipped. The 2 failures are the gitleaks download (the proxy returned HTTP 504) and the scan step that needs it; a gitleaks scan of origin/main..HEAD run by hand found no leaks. Tests: 9,059 passed, 2 skipped.
• Mutation runs on the new lines: replacing unscored.get(name, 0) + routed["n_failed"] with routed["n_failed"] fails the new test ({'alpha': 1} != {'alpha': 2}, 1 failed, 55 passed). Deleting the args.repeats > 1 check fails only test_all_vs_all_refuses_repeats_it_would_not_make (1 failed, 55 passed).

Checked, no issues

• --repeats: the default is the int 1 (positive_int), so the comparison is safe. The refusal fires before any model call and before any --out file is written; the test asserts both.
• Docs and changelog: the --artifact, --queries, --queries-from-headings and --repeats refusals in skill-novelty.md and the all-vs-all fragment match _check_all_vs_all_args.
• Growth baseline: novelty.py entry raised to 1176 with a rationale; check_growth.py passes in ci-local.

@napetrov's question on whether two differently named skills with identical text should be one tied group waits for the author.