Skip to content

fix(agent-team): mark finalize-gate bypass on the final turn instead of delivering silently - #48

Open
KaiOnCode wants to merge 3 commits into
ApodexAI:mainfrom
KaiOnCode:fix/finalize-gate-bypass-visibility
Open

KaiOnCode wants to merge 3 commits into
ApodexAI:mainfrom
KaiOnCode:fix/finalize-gate-bypass-visibility

Conversation

@KaiOnCode

Copy link
Copy Markdown

Related to #19 (delivery half; the prompt/tool blocked/notes mismatch needs a direction decision and would be a separate PR).

Summary

BareTextFinalizeObserver accepts the plain-text answer on the final turn even when finalize_gate blocks it — the deliberate "never lose the answer to max_turns" fallback — but latched it with no trace. A run whose task board still has open items was delivered exactly like a clean success: only an INFO line (gate=bypassed(last_turn)) recorded that the gate had said Cannot finish. This is the delivery half of #19: the reported run ended with "WAF rules and Docker image delivered / 100% blocked" while three core board items were still open.

Changes

  • when the last-turn bypass fires, set metadata["finalize_gate_bypassed"] to the full gate message (machine-readable marker)
  • append a user-visible note to the latched answer naming the still-unresolved board items: > ⚠ Unfinished work at submission: task-board item(s) still unfinished: ...
  • mid-run rejection and clean-pass behavior unchanged — both locked by tests
  • CHANGELOG entry under Unreleased → Fixed

Verification

  • uv run pytest -q tests/test_bare_text_finalize_gate.py — 3 passed; the bypass-marking test is red on unpatched main
  • uv run ruff check frontier_agent/ apodex/ benchmarks/ workflows/ plugins/ deploy/ tools/ scripts/ — passed
  • uv run pyright — 0 errors
  • uv run python tools/import_smoke.py --stage 1 — 289/289 modules; --stage 2 — 338/338 modules
  • uv run python tools/check_symbols.py / tools/check_lazy_exports.py — OK
  • uv run pytest -q — 1738 passed (run twice, before and after final diff cleanup)

AI assistance was used during implementation. I reviewed the final diff and ran the verification listed above.

…of delivering silently

BareTextFinalizeObserver accepts the answer at turn >= max_turns - 1 even
when finalize_gate blocks it (the never-lose-the-answer fallback) but
latched it with no trace, so a run with open task-board items was
delivered exactly like a clean success. Set metadata
["finalize_gate_bypassed"] and append a user-visible note naming the
still-unresolved items. Mid-run rejection and clean-pass behavior are
unchanged and locked by tests.

Related to ApodexAI#19
@zhanghanduo

Copy link
Copy Markdown
Collaborator

Found two issues that should be addressed before merging:

  1. [P2] Propagate the bypass marker into the workflow output — workflows/agent_team/observers/bare_text_finalize.py:67 sets finalize_gate_bypassed in loop-local metadata, but main_agent_node does not copy that field into its returned workflow output. Consumers still receive answer_status="complete" without the machine-readable bypass marker. Please propagate the marker into the workflow result and add a test at that boundary.

  2. [P2] Preserve the warning through reporter finalization — workflows/agent_team/observers/bare_text_finalize.py:68 appends the warning after the entire answer. When an answer ends with a References section and the reporter retains that section, finalize_report_with_canonical_references strips the section and everything following it, including the warning. I reproduced this locally by passing an answer with inline citations, a trailing References section, and the appended warning through that finalizer: the warning was present before finalization and absent afterward. Please append the warning after reporter finalization or explicitly preserve it through reference cleanup.

Validation: all three tests in tests/test_bare_text_finalize_gate.py pass. They cover the observer, but not these downstream delivery boundaries.

…reporter finalization

Review on ApodexAI#48: the marker died in loop-local metadata (consumers only
saw answer_status="complete"), and the warning appended after a trailing
References section was stripped by the reporter's citation cleanup.

- publish finalize_gate_bypassed + finalize_gate_warning from
  main_agent_node and list them in both specs (main output_fields,
  reporter include_fields/output_fields)
- the observer now stores a ready-to-append warning built while the
  task board is still live instead of mutating the latched answer
- delivery nodes append the warning after finalization: main_agent for
  the coordinator's own answer, agent_team_reporter after
  _run_fast_reporter returns (References cleanup has already run)
- append_bypass_warning is idempotent
- five new tests cover both delivery boundaries and document the strip
  hazard, including strip_trailing_references' 30% heading guard
@KaiOnCode

Copy link
Copy Markdown
Author

Thanks @zhanghanduo — both points are addressed in 4f4eb5f.

1. Marker propagation into the workflow output
main_agent_node now publishes finalize_gate_bypassed alongside a new finalize_gate_warning (the ready-to-append warning text), and both keys are listed in spec.py / spec_report.py: main_agent.output_fields, agent_team_reporter.context_policy.include_fields, and agent_team_reporter.output_fields — so the marker travels with answer_status and survives into the reporter's state.

2. Warning preserved through reporter finalization
Restructured exactly along your "append after finalization" option: the observer no longer mutates the latched answer. It stores the gate message plus a warning built while the task board is still live (the board is cleared before output assembly). The visible text is appended idempotently at the delivery boundaries:

  • main_agent_node for the coordinator's own answer (covers reporter-off and reporter fail-open return {} paths),
  • agent_team_reporter after _run_fast_reporter returns, i.e. after finalize_report_with_canonical_references has run.

One detail from reproducing your case: strip_trailing_references refuses to cut when the References heading starts before 30% of the body, so the loss only triggers on realistically long reports — toy samples keep the warning but then emit a duplicate References block. The hazard test uses a full-length body and documents the guard.

Validation

  • uv run pytest -q tests/test_bare_text_finalize_gate.py — 8 passed (5 new: marker+warning storage, both delivery boundaries, spec plumbing ×2 pipelines, strip hazard, idempotency)
  • uv run pytest -q — 1743 passed
  • uv run ruff check <CI dirs> — passed · uv run pyright — 0 errors
  • tools/import_smoke.py --stage 1/2 — 289/289, 338/338 · check_symbols.py / check_lazy_exports.py — OK

AI assistance was used during implementation. I reviewed the final diff and ran the verification listed above.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants