Skip to content

fix(scanner): attribute a marker to the test it annotates (REQ-326, #892, #787) - #897

Open
avrabe wants to merge 1 commit into
mainfrom
fix/req-326-marker-attribution
Open

fix(scanner): attribute a marker to the test it annotates (REQ-326, #892, #787)#897
avrabe wants to merge 1 commit into
mainfrom
fix/req-326-marker-attribution

Conversation

@avrabe

@avrabe avrabe commented Sep 6, 2026

Copy link
Copy Markdown
Contributor

Closes #892. Closes #787.

find_enclosing_function only ever scanned backwards, so a marker written in
the conventional place — on the line above the #[test] it annotates — was
attributed to the function before it. Reported twice, from meld (#892) and
earlier as #787.

#[test]
fn first_unrelated_test() {}

// rivet: verifies REQ-001          → reported: first_unrelated_test
#[test]
fn the_test_that_actually_verifies_it() {}

The damage is narrow and bad

The requirement mapping stays correct — rivet verify still advances, and
coverage percentages are unaffected. What's wrong is the evidence line, which
names a different test than the one that verifies the requirement.

For a reader auditing the right-hand side of the V, that line is the product.
Following it leads to a test that does not test the thing.

Why the fix is bounded

Forward walk, but only across lines that may legitimately separate a marker from
the item it annotates: attributes, comments, blank lines. A marker inside a
body has real code on the next line, so the walk stops and the existing backward
scan returns the enclosing function — which is the shell convention REQ-319
depends on.

Negative-controlled both ways, which is what shows the bound is load-bearing:

remove the forward walk     1 failed   (the reported bug returns)
make the walk unbounded     2 failed   (in-body case AND the shell case)

A mistake in this commit, and what caught it

Inserting these tests spliced a doc comment into the middle of REQ-319's own
doc comment
, orphaning its rivet: verifies line onto the wrong function.

cargo test passed. clippy passed. Doc comments concatenate harmlessly, so
nothing in the Rust toolchain could see it. Only rivet coverage --tests caught
it — by reporting REQ-319 against a test with nothing to do with shell scanning.

The tool found a defect the compiler could not. That's worth more than the fix.

Gates

cargo fmt --check · clippy --all-targets -D warnings on 1.97.0 ·
cargo test --workspace (exit 0, 66 ok) · full cli_commands (189 passed) ·
rivet validate · rivet docs check · yamllint — all exit 0.
REQ-326 → verified, 2 test markers, correctly attributed.

, #787)

`find_enclosing_function` only ever scanned BACKWARDS, so a marker written in
the conventional place — on the line above the `#[test]` it annotates — was
attributed to the function before it. Reported twice, from meld (#892) and
earlier as #787. Three lines reproduce it:

  #[test]
  fn first_unrelated_test() {}

  // rivet: verifies REQ-001        -> reported first_unrelated_test
  #[test]
  fn the_test_that_actually_verifies_it() {}

The damage is narrow and bad. The requirement MAPPING stays correct, so `rivet
verify` still advances and coverage percentages are unaffected. What is wrong is
the EVIDENCE LINE, which names a different test than the one that verifies the
requirement — and for a reader auditing the right-hand side of the V, that line
is the entire product. Following it leads to a test that does not test the thing.

Fixed by walking FORWARD first, but only across lines that may legitimately
separate a marker from the item it annotates: attributes, comments, blank lines.
The bound is what makes it correct rather than merely different. A marker inside
a function body has real code on the next line, so the walk stops there and the
existing backward scan returns the enclosing function — which is the shell
convention REQ-319 depends on.

Negative-controlled both ways at --lib scope, which is what shows the bound is
load-bearing rather than decorative:

  remove the forward walk     1 failed  (the reported bug returns)
  make the walk unbounded     2 failed  (in-body case AND the shell case)

Worth recording how a mistake in this commit was caught. Inserting these tests
spliced a doc comment into the middle of REQ-319's own doc comment, orphaning
its `rivet: verifies` line onto the wrong function. `cargo test` and `clippy`
both passed — doc comments concatenate harmlessly — and only
`rivet coverage --tests` caught it, by reporting REQ-319 against a test that has
nothing to do with shell scanning. The tool found a defect the compiler could
not see, which is the case for the tool existing.

Confirmed with cargo fmt --check, clippy --all-targets -D warnings on 1.97.0,
cargo test --workspace (exit 0, 66 ok, 0 failed), the full cli_commands suite
(189 passed), rivet validate, rivet docs check, yamllint — all exit 0.

Implements: REQ-326
Verifies: REQ-326
@codecov

codecov Bot commented Sep 6, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.72727% with 1 line in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
rivet-core/src/test_scanner.rs 97.72% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

📐 Rivet artifact delta

Change Count
Added 1
Removed 0
Modified 0
Downstream impacted (depth ≤ 5) 0

Graph

graph LR
  REQ_326["REQ-326"]:::added
  classDef added fill:#d4edda,stroke:#28a745,color:#155724
  classDef removed fill:#f8d7da,stroke:#dc3545,color:#721c24
  classDef modified fill:#fff3cd,stroke:#ffc107,color:#856404
  classDef overflow fill:#e2e3e5,stroke:#6c757d,color:#495057,stroke-dasharray: 3 3
Loading
Added
  • REQ-326

📎 Full HTML dashboard attached as workflow artifact rivet-delta-pr-897download from the workflow run.

Posted by rivet-delta workflow. The graph shows only changed artifacts; open the HTML dashboard (above) for full context.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant