Skip to content

Summarize with the agent's tools bound, and never resume from an empty summary - #42

Open
ericeil wants to merge 1 commit into
masterfrom
eric/summary-nonempty
Open

ericeil wants to merge 1 commit into
masterfrom
eric/summary-nonempty

Conversation

@ericeil

@ericeil ericeil commented Nov 6, 2026

Copy link
Copy Markdown
Contributor

Problem

The summarizer sent the history to the unbound model. After a long run of tool calls, Opus 5.5 answers a bare summarization request by ending its turn with no text. On a cloud CVLR run (Solana stake program, Withdrawal author) the author compacted three times. The three summarize calls returned 27, 1 and 24 output tokens, and each time the author resumed from BEGIN SUMMARY: END SUMMARY. Each call also wrote ~517k tokens to the cache, since without the tool definitions the prefix differs from the agent's.

Replaying the first real request, loaded from the checkpoint:

request result
unbound (production) end_turn, 1 output token, empty, both times
tools bound tool_use: calls memory, no text
either, plus "reply in plain text" a 4,126- / 4,396-token summary

Change

  • The summarizer calls the tool-bound model, so the request shares the agent's cached prefix.
  • Every summarization request ends with "Do not call any tool in this turn. Reply with the summary as plain text."
  • A reply with no text is retried once (SUMMARY_ATTEMPTS = 2). If it is still empty, the history is kept and the next turn summarizes again, with a warning naming the stop reason, the tool calls and the output tokens. A summarization that raises is now logged; before, it was dropped silently.
  • _stitch_sync_impl / _stitch_async_impl let a node generator make more than one LLM request, which the retry needs. Single-request nodes behave as before.

Testing

  • tests/test_summarizer.py: the request asks for plain text, a text reply replaces the history, an empty reply is retried, and two empty replies keep the history.
  • Full graphcore suite: 276 passed. AutoProver's suite with this branch checked out: 2,659 passed. pyright is clean on the changed files in both.

🤖 Generated with Claude Code

…y summary

The summarizer sent the history to the unbound model. After a long run of tool calls,
Opus 5.5 answers that request by ending its turn with no text: a CVLR author on the Solana
stake program compacted three times in one run, got 27, 1 and 24 output tokens back, and
resumed each time from an empty summary. Replaying the first request reproduced it every
time. With the tools bound the model calls `memory` instead; with either, a closing
instruction to answer in plain text produces the summary (4,126 and 4,396 tokens).

- The summarizer calls the tool-bound model, so the request shares the agent's cached
  prefix. Each of those compactions wrote ~517k tokens to the cache instead.
- Every summarization request ends with an instruction not to call a tool.
- A reply with no text is retried once. If it is still empty, the history is kept and the
  next turn summarizes again, with a warning naming the stop reason and any tool calls.
  A summarization that raises is logged rather than dropped silently.
- A node generator may now make more than one LLM request.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
ericeil added a commit to Certora/AutoProver that referenced this pull request Nov 6, 2026
… summary

Pins graphcore at f4221b6, the head of Certora/graphcore#42 (open). Re-pin to the merge commit
once it lands. The bump also brings in graphcore master's #41 (bounded grep_files output).

On the cloud stake run all three compactions of the Withdrawal author returned no text (27, 1 and
24 output tokens), and the author resumed each time from an empty summary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant