Skip to content

fix: preserve successive CLI regression findings - #89

Merged
CodewithJha merged 1 commit into
CodewithJha:mainfrom
fatihcvs:fix/preserve-regression-findings
Sep 24, 2026
Merged

CodewithJha merged 1 commit into
CodewithJha:mainfrom
fatihcvs:fix/preserve-regression-findings

Conversation

@fatihcvs

Copy link
Copy Markdown
Contributor

Summary

Fixes #68. Successive verified findings now use the first free cli_discovered_violation[_N].json path instead of overwriting earlier regressions. Exclusive file creation protects against races between writers; the artifact name, returned regression ID and event path follow the allocated filename.

The first filename remains compatible with existing samples and hosted-sync fixtures. The CLI guide documents numbering and intentional replacement via backup/removal/re-run; no new --force option is introduced.

Type of change

  • Bug fix
  • Tests
  • CLI / DX

Checklist

  • Read CONTRIBUTING.md
  • Linked related issue
  • Core stays free of framework SDK imports
  • Tests added for changed behavior
  • Documentation matches implementation
  • One concern per PR
  • Authorized local testing only

Test plan

pytest tests/unit -q: 378 passed on Windows / Python 3.12.14 with uv on PATH and UTF-8 enabled.

The two new cases failed before the fix. They run two actual local campaigns against the in-process demo adapter, verify distinct artifacts and unchanged previous bytes (including a pre-existing user file), check event/ID consistency, and replay both findings through run_tests. Model mutation is explicitly disabled in these tests; no live provider testing is claimed.

@vercel

vercel Bot commented Sep 24, 2026

Copy link
Copy Markdown

@fatihcvs is attempting to deploy a commit to the priyanshu's projects Team on Vercel.

A member of the Team first needs to authorize it.

@CodewithJha CodewithJha left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @fatihcvs — this hits #68 / MUT-020 cleanly: successive findings allocate cli_discovered_violation[_N].json via exclusive create, so prior regressions and pre-existing user files are preserved. The first filename stays compatible; mutiny test discovers both; replace is documented without a new --force flag. GitHub Actions green on 3.11/3.12; ignoring Vercel auth.

@CodewithJha
CodewithJha merged commit fea83c2 into CodewithJha:main Sep 24, 2026
5 of 6 checks passed
@CodewithJha

Copy link
Copy Markdown
Owner

Merged — thank you @fatihcvs. Exactly the kind of CLI reliability fix Mutiny needs: successive verified findings keep their own regression files, exclusive create avoids races, and the original cli_discovered_violation.json name stays compatible.

If you'd like to continue working on Mutiny, these would be good related areas to look at:

  • #57 — mutiny run error handling still needs the same CLI care
  • #59 — interrupted --hosted ingest can leave campaigns stuck running
  • #69 — second in-process project load can reuse the first project's agent module

No pressure — only if you want another one.

CodewithJha pushed a commit that referenced this pull request Sep 24, 2026
Keep the contributor-facing audit index accurate now that MUT-017/020/023/024/025/037 shipped.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] MUT-020: Every CLI finding overwrites cli_discovered_violation.json

2 participants