[antonwolfy]

Summary

Follow-up to the review of #103 (thread). napetrov fixed part of my non-blocking point and left the rest for a follow-up, which I agreed to. On main, each mutant of scripts/audit_test_quality.py below passes all 133 tests of the gate's test file. Only tests change.

Changes

tests/test_audit_test_quality.py:

• test_a_body_with_other_inputs_is_not_a_duplicate gets five pairs of tests whose signatures differ in only one way: keyword-only fixtures, positional-only fixtures, keyword-only defaults, *args or **kwargs. Dropping any of those parts from the duplicate identity (_duplicates) passed before.
• test_every_file_pytest_collects_is_audited also runs with the files two levels down, so glob instead of rglob for either pattern in audit_tree fails it.
• test_the_report_marks_only_an_exempt_test_as_exempt: report() printing the (exempt) suffix never, or always, passed before.
• test_a_patched_test_that_expects_a_raise_is_not_a_mock_echo: napetrov also listed three single-flag mutants. Dropping not t.raises from mock_echo still passed. The other two, _is_weak with any over every BoolOp and type_only ignoring assert_* calls, already fail a test on main.

Risk

None: tests only, so no changelog fragment. #231 edits the same test file in other places; the two branches merge cleanly.

Testing

On a Linux review host, at 537be67 (on main 54ce188):

• make ci-local: 29 passed, 0 failed, 7 skipped (the Docker catalog smoke, test-latest, the cli-surface packaging step, conda-forge-deps without micromamba, docs deploy, the pip-audit report, semgrep). 8,208 tests passed, 2 skipped.
• Each of the 10 mutants of scripts/audit_test_quality.py fails one test of the file: kw_defaults, vararg, kwarg, kwonlyargs or posonlyargs dropped from the identity; glob for test_*.py or for *_test.py; the suffix never or always printed; not t.raises dropped from mock_echo. On main all 10 pass its 133 tests.
• With the four follow-up branches merged together on main 54ce188 (0a5717c): the full suite (8,232 passed, 2 skipped), the growth check, mypy and ruff pass.

Checklist

• Tests added/updated and pytest passes
• ruff, mypy, and schema checks pass
• Docs updated (or N/A) — see the docs-drift checklist in CONTRIBUTING.md
• No ad-hoc run artifacts committed
• New files carry the SPDX license header (or are exempt in .github/.licenserc.yaml)

[codecov-commenter]

⚠️ Please install the to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 92.24%. Comparing base (54ce188) to head (537be67).
⚠️ Report is 3 commits behind head on main.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #232      +/-   ##
==========================================
- Coverage   92.25%   92.24%   -0.01%     
==========================================
  Files         239      239              
  Lines       29937    29937              
  Branches     4918     4918              
==========================================
- Hits        27618    27616       -2     
- Misses       1632     1633       +1     
- Partials      687      688       +1     

see 1 file with indirect coverage changes

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[antonwolfy]

@exai-bot /review

[exai-bot]

Note

AI-generated review of 537be6742dde, requested by @antonwolfy, using the default review skill. It can be wrong: verify a finding before acting on it. React with a thumbs up or down on a finding to tell the bot whether it helped.
Model us.anthropic.claude-sonnet-5-5, effort default, 5 of 80 turns, 0.4 of 40 minutes, 59k in / 2k out tokens.

This PR only adds tests to test_audit_test_quality.py. They cover the duplicate identity (keyword-only and positional-only fixtures, keyword-only defaults, *args, **kwargs), nested test files, the (exempt) report suffix, and not t.raises in mock_echo. I read each new case against scripts/audit_test_quality.py (_duplicates, audit_tree, report, mock_echo). Each one would fail under the mutant it targets, and none looks vacuous. I found no defects. I could not run pytest here because it can't be installed in the sandbox.

No issues found.

Not verified

I did not run the test file or the 10 mutants. Their results rest on my reading of the code and on the PR's description.

[Copilot]

🟢 Approval recommended

The isolated test-only changes cover the intended auditor behavior, with no blocking issues identified.

No open issues 🎉

What changed in this PR

Adds regression coverage for the test-quality auditor introduced in #103, without changing production code or auditor behavior.

Changes:

• Covers five signature differences that must prevent duplicate-test flags.
• Verifies recursive discovery for both test-file naming patterns.
• Checks exemption suffixes in reports and ensures expected exceptions prevent mock_echo flags.

File	Description
dokimi/​tests/​test_audit_test_quality.py	Adds focused regression cases for duplicate identity, nested discovery, reporting, and exception assertions.

_{Review effort: Balanced}

[napetrov]

Review of 537be67.

Summary: tests only. The new cases for the duplicate identity (keyword-only and positional-only fixtures, keyword-only defaults, *args, **kwargs), nested test files, the (exempt) suffix and not t.raises match what the PR body says they pin. No fixes pushed.

Test evidence (local, on 537be67):

• uv run pytest dokimi/tests/test_audit_test_quality.py: 141 passed.
• ruff check on the touched file: clean.
• The audit of the repo's own tests is unchanged: only the existing exempt test is reported.

Open items: none. As the body says, #231 edits the same file. Whichever lands second should be rerun against the other.

Verdict: ready from my side.

[vchamarthi]

Tests match the auditor's current behavior at the head; no issues found.