Skip to content

chore(coverage): measure and print statement coverage, no threshold - #53

Merged
TAIPANBOX merged 2 commits into
mainfrom
chore/coverage-measurement
Sep 16, 2026
Merged

TAIPANBOX merged 2 commits into
mainfrom
chore/coverage-measurement

Conversation

@TAIPANBOX

Copy link
Copy Markdown
Owner

What

No coverage percentage was measured or gated anywhere in this repository:
no coverage/pytest-cov dependency, no CI step read one. This adds the
measurement without changing behaviour: no dependency swap, no algorithm
touched, no threshold that can fail a build.

  • pytest-cov added to the dev extra (pyproject.toml).
  • [tool.coverage.run] (source = engram, tests/* omitted) and
    [tool.coverage.report] (no fail_under) in pyproject.toml.
  • CI's test job now runs pytest --cov=engram --cov-report=term-missing
    instead of plain pytest, printing the TOTAL line on every push, across
    the 3.11/3.12/3.13 matrix, at no extra cost (same pytest invocation the
    job already made).
  • README states the measured figure next to the existing test-coverage
    heading, with the command that produced it.

A deliberate limit, not an oversight

scripts/readme-numbers.sh does not recompute this number the way it
recomputes the test count. The test count uses --collect-only
specifically to stay near-instant (about 1.2s measured); a full
pytest --cov run here takes about 53-57s. Wiring coverage into that
script's auto-check would multiply its cost 5-6x inside
gates-have-teeth.sh, which replays readme-numbers.sh once per fault case
it exercises (roughly 5 minutes added to what is today a ~7 second run).
Measured both ways before deciding: gates-have-teeth.sh with the coverage
recompute wired in versus without it is the same 8-case harness, at 7.3s
total either way once the recompute was left out. The comment in
readme-numbers.sh names this as a call for the project owner to override,
not a closed decision.

Report

Tier:       T0/T1, docs and tooling: no product code path changed, no
            default, weight or dependency behaviour touched
Scenarios:  none; no behaviour to describe in Given/When/Then
Red first:  not applicable, no defect and no new assertion path
Tests:      0 new; full suite unaffected, 525 passed / 1 skipped (with the
            encryption extra installed locally to measure both environments,
            see below), matching pre-change behaviour
Gate:       readme-numbers.sh explicitly does NOT gate this number (see
            above); the number is printed, not enforced, per the task
Metrics:    ruff check: all checks passed. ruff format --check: 59 files
            already formatted. mypy engram: could not run in this .venv,
            see NOT proven (same pre-existing issue as the companion PR).
            pytest: 525 passed, 1 skipped (test_manifest, console script
            not on PATH). Coverage @measured 2026-09-16,
            `.venv/bin/pytest --cov=engram --cov-report=term-missing`:

              89% total (2679 statements, 293 uncovered) on a plain install
              89% total (2679 statements, 282 uncovered) with the encryption
                 extra installed (same percentage; the extra's tests lower
                 the raw miss count by exercising Engram(key=...))

            Five least-covered files (plain install):
              engram/__main__.py             0%  (2 stmts, 2 uncovered)
              engram/benchmarks/__main__.py  0%  (2 stmts, 2 uncovered)
              engram/benchmarks/cli.py      46%  (183 stmts, 98 uncovered)
              engram/adapters/llamaindex.py 62%  (94 stmts, 36 uncovered)
              engram/adapters/langchain.py  80%  (64 stmts, 13 uncovered)

            The two 0% files are one-line `python -m engram` / `python -m
            engram.benchmarks` shims never exercised because the suite
            imports the package directly; the rest are optional
            integration/benchmark-CLI surfaces with lighter test investment
            than the core library.
Ran it:     ruff check ., ruff format --check ., pytest (plain and with
            --cov, twice: plain install and with the encryption extra
            installed), ./scripts/no-network-at-write.sh (OK),
            ./scripts/readme-numbers.sh (OK, ~1.2s, confirmed unchanged
            cost), ./scripts/local-first.sh (OK),
            ./scripts/gates-have-teeth.sh (all 8 cases OK, ~7.3s, tree left
            clean). All from the repo's own .venv, activated.
NOT proven: mypy engram could not be run in this .venv for the same
            pre-existing, unrelated reason as the companion PR (Python 3.14
            here, outside the 3.11-3.13 CI matrix; mypy 2.3.1 fails to parse
            a PEP 695 construct in the installed numpy 2.5.3 stub
            regardless of the configured target python_version); verified
            against unmodified main via git stash. No threshold is set on
            the coverage number, so nothing here stops it from silently
            dropping on a future change; that is an explicit, named
            decision left to the project owner (see above), not a gap in
            this change. The five least-covered files above are not
            improved by this PR; only the practice of measuring them is
            added.

🤖 Generated with Claude Code

TAIPANBOX and others added 2 commits September 16, 2026 14:02
No coverage percentage was measured or gated anywhere in this repo:
no coverage/pytest-cov dependency, no CI step read one (recorded as a
gap in the architecture dossier's read of this repo). Per the
project's standing rule that test coverage is maximised and reported,
not merely available, add the measurement without changing behaviour.

- pytest-cov added to the dev extra.
- [tool.coverage.run]/[tool.coverage.report] in pyproject.toml: source
  is the engram package, tests/ omitted, no fail_under. Whether to gate
  on a threshold is a separate decision for the project owner; this
  only makes the number visible.
- CI's test job now runs `pytest --cov=engram --cov-report=term-missing`
  instead of plain pytest, so the TOTAL line prints on every push
  across the 3.11/3.12/3.13 matrix, at no extra cost: it is the same
  pytest invocation the job already made.
- README states the measured figure next to the existing test-coverage
  heading, with the command that produced it.

scripts/readme-numbers.sh deliberately does NOT recompute this number
the way it recomputes the test count: that check uses --collect-only
to stay near-instant, and a full pytest --cov run costs about 50
seconds here. Wiring it in would multiply that cost 5-6x inside
gates-have-teeth.sh, which replays readme-numbers.sh once per fault
case it exercises. Measured: readme-numbers.sh unchanged in
scope stays at about 1.2s; a full pytest --cov=engram
--cov-report=term-missing run here takes about 53s to 57s. Left as a
comment in the script and as a decision for the project owner rather
than made unilaterally.

@measured 2026-09-16, this .venv (.venv/bin/pytest --cov=engram
--cov-report=term-missing): 89% statement coverage on a plain install
(2679 statements, 293 uncovered) and 89% with the encryption extra
installed (282 uncovered; the extra exercises the Engram(key=...) path
so its raw miss count is lower, though the two round to the same
percentage today).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ve no gate yet

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@TAIPANBOX
TAIPANBOX merged commit 240a4d9 into main Sep 16, 2026
6 checks passed
@TAIPANBOX
TAIPANBOX deleted the chore/coverage-measurement branch September 16, 2026 14:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant