fix: preserve matched conditions in require_args evidence - #84
Conversation
|
@Sonike is attempting to deploy a commit to the priyanshu's projects Team on Vercel. A member of the Team first needs to authorize it. |
CodewithJha
left a comment
There was a problem hiding this comment.
Thanks @Sonike — this hits #79 / MUT-037 cleanly: the require_args success path now tracks whether any relevant call matched when, so a compliant issue_refund(amount=250, approved=True) reports matched_when=True instead of the hardcoded False. Multi-call ordering, later violations, unmatched when, and absent tools stay correct. GitHub Actions green on 3.11/3.12; ignoring Vercel auth.
|
Merged — thank you @Sonike. Exactly the kind of focused Core evaluator fix Mutiny needs: compliant If you'd like to continue working on Mutiny, these would be good related areas to look at:
No pressure — only if you want another one. |
Summary
require_argscall with a matching condition incorrectly reportedmatched_when=False. Track whether any relevant call satisfies the condition so successful evidence reportsTrue, including when later calls do not match.Type of change
Checklist
Fixes #N) or Discussion when applicableuv run pytest tests/unit -q)Test plan
Validated with Python 3.12.13 and the offline sample mode (
MUTINY_SAMPLE_OFFLINE=1), with isolated temporary database paths:uv run --offline pytest tests/unit/test_policy_evaluator.py -q --tb=short: 5 failures before the fix; 123 passed after.uv run --offline pytest tests/unit -q --tb=short: 380 passed, including package build/content checks.uv run --offline pytest tests/integration tests/reliability -q --tb=short: 140 passed (one existing Starlette deprecation warning).git diff --check: passed.