fix(cli,core): handle adapter/config errors and unify missing policy handling (#57)

[littfed]

Summary

Resolves #57 (MUT-003, MUT-014, MUT-034):

• Prevents Python tracebacks on adapter import / load failure in mutiny run; exits 2 with clean hint pointing at mutiny init.
• Validates CampaignConfig before printing the search banner, cleanly exiting 2 on invalid parameters or malformed YAML.
• Catches adapter/execution runtime exceptions in CampaignEngine.run and translates them into CampaignResult(status="error") (non-zero CLI exit, no process crash).
• Preserves generations_completed > 0 and non-None best candidate when ToolsNotObservableError is raised after scoring candidates.
• Replaces dead FileNotFoundError handling in test_cmd.py with PolicyFileNotFoundError, providing identical "missing policy / run mutiny init" error copy across mutiny run and mutiny test.

Verification

• Added unit tests in tests/unit/test_mutiny_cli_run.py, tests/unit/test_campaign.py, and tests/unit/test_policy_load.py.
• Ran full test suite: 575 passed.
• Verified local CLI smoke test in examples/openai_support_agent.

[vercel]

@littfed is attempting to deploy a commit to the priyanshu's projects Team on Vercel.

A member of the Team first needs to authorize it.

[CodewithJha]

Thanks @littfed — this is a serious attack on #57 (MUT-003 / MUT-014 / MUT-034), not a drive-by. I checked out 4803de2 and ran it.

What already lands

• CampaignConfig is built before the search banner; population_size: 100 exits 2 with invalid mutiny.yaml and no traceback.
• Dedicated PolicyFileNotFoundError + identical mutiny init copy on run and test. Dead FileNotFoundError handler is gone.
• mutiny init && mutiny run in an empty project exits 2 with an adapter hint (no raw traceback).
• Sample mutiny run still finds the refund-limit hit and writes a regression.
• Unit tests for the three findings are real. GitHub Actions (3.11/3.12, cli-smoke, package-build, web-build) are green. Vercel authorize noise is ignored.
• AGENT_REF wording correctly left to #27.

Not merging yet. The PR’s contract is “adapter/runtime failures become status=error, no traceback.” Two paths I reproduced still violate that:

1. generations_done is unbound if _initial_population() raises — including on_event during seed emit. The new except then raises UnboundLocalError and the original error is lost. Initialize generations_done = 0 before the try (same for both ToolsNotObservableError and Exception handlers).
2. CLI minimize after status=error + violated=True. With stop_on_first_violation: false, a later RuntimeError returns an error result that still has a violating best. _run_local then calls _maybe_minimize_and_save on the dead adapter and the traceback comes back (RuntimeError: sdk exploded after first candidate). Skip minimize when result.status == "error" (or wrap it so minimize failures cannot abort the process).

Please add unit coverage for both (event-handler raise during seed emit; CLI run with stop-on-first off + crash after a hit). After those two, this is merge-ready for #57 — do not fold in #27 / #70.

GitHub Actions are sufficient; ignore the Vercel authorize check.

[CodewithJha]

Blocking: generations_done is assigned after _initial_population(). If seed construction or a CANDIDATE_CREATED on_event raises, this line (and the ToolsNotObservableError twin above) raises UnboundLocalError and swallows the original exception.

I reproduced it with an on_event that raises on EventType.CANDIDATE_CREATED — engine.run() escaped with UnboundLocalError: cannot access local variable 'generations_done'.

Fix: generations_done = 0 before the try (the 1 if all_scored else 0 fallback can stay). Please add a unit test for this path.

[CodewithJha]

Preserving best / violated here is the right MUT-014 call for metrics. It does change CLI behavior: _run_local still does if result.violated and result.best is not None: _maybe_minimize_and_save(...) with no status != "error" guard.

Reproduced with stop_on_first_violation: false and an adapter that scores a refund violation then raises RuntimeError on the next candidate:

✓ Local finished: status=error ... violated=True candidates=1
  minimizing exploit …
RuntimeError: sdk exploded after first candidate

That reintroduces the traceback this PR is supposed to kill (default stop_on_first_violation: true hides it). In run_cmd.py, skip minimize when result.status == "error" — the adapter is not trustworthy. A CLI unit test for that path would lock it.

[CodewithJha]

Non-blocking: this is what makes mutiny init && mutiny run exit 2 instead of falling into Core status=error on lazy agent_ref load. Fine for Adapter #1.

If you touch this file for the minimize guard, a one-line comment that this is an eager resolve for OpenAIAgentsAdapter (not part of TargetAdapter) would help the next reader. Do not expand into #27 AGENT_REF copy.

[littfed]

Thanks for the fast and specific review @CodewithJha!

Updated in c2db030:

1. generations_done unbound handling:

   • Initialized generations_done = 0 before the try block in CampaignEngine.run, and safely wrapped CAMPAIGN_ERROR emission.
   • Added unit test test_event_handler_raise_during_seed_emit_handled_as_status_error in tests/unit/test_campaign.py verifying that an exception raised by on_event during seed emit cleanly returns CampaignResult(status="error") with generations_completed=0 and no UnboundLocalError.

2. Skip CLI minimization on status == "error":

   • In _run_local (run_cmd.py), added an explicit check result.status != "error" before attempting _maybe_minimize_and_save, and additionally wrapped minimization in a try/except guard so unexpected minimize failures cannot abort the process.
   • Added unit test test_stop_on_first_violation_false_and_crash_after_hit_skips_minimize_and_exits_1 in tests/unit/test_mutiny_cli_run.py covering stop_on_first_violation: false followed by a runtime crash after an initial hit, ensuring minimization is skipped and CLI exits 1 without traceback.

Verified both new tests and smoke run in examples/openai_support_agent — all passing cleanly.