fix(agent): heal threads left with an unanswered tool call

[ciaransweet]

Problem

A run cancelled while a tool is running leaves the model's tool call in the checkpoint with no result. This happens when the client closes the stream or a pod is rolled mid-turn. Providers then reject every later turn on that thread, and nothing the user sends can fix it.

Seen in production on the ECMWF deployment (ecmwf/dss-agentic-ai-services#231):

• The browser closed /runs 8 seconds into a search_knowledge call, and uvicorn cancelled the run (Cancelled via cancel scope … by RequestResponseCycle.run_asgi()).
• Every later message on the thread got Mistral 400 Not the same number of function calls and responses (code 3230).

Fix

New mcp_agent.interrupted_tool_calls: a before_agent middleware that closes each unanswered tool call with an error ToolMessage placed straight after it. It writes that back to the checkpoint, so the thread heals once, on its next turn, including threads that are already broken.

• with_session_state always wires it in, after StateCaptureMiddleware and before the host's own middleware.
• build_agent with session state off (MCP_AGENT_STATE=0) skips with_session_state but still keeps conversations, so it carries the repair as well.
• docs/CONSUMING.md says both bundled agents carry it, and tells a host assembling its own checkpointed agent (section 4b) to add it to middleware.
• A thread paused on interrupt() is never touched. A message on a paused thread is refused before the graph runs, and a resume re-enters the paused node without going through before_agent.

Proof

• tests/mcp_agent/test_interrupted_tool_calls.py really cancels a run mid-tool against a model that validates history the way Mistral does.
  • With a plain create_agent, the next turn raises the 3230 error.
  • Through with_session_state, the next turn answers, and the repaired checkpoint has the placeholder right after the call.
• Live: the production thread's checkpointed messages replayed against mistral-small-latest return the 400 as stored. Repaired, they succeed, and the model reissues the interrupted search_knowledge call on its own.

🤖 Generated with Claude Code

[j08lue]

Ok, I guess.

[j08lue]

Better parameterise this ID. But nvm.

[j08lue]

Interesting static detail. So this expectation is provider-dependent?

This will likely break when we use another provider.

[ciaransweet]

In that they report it differently, but the high level expectation is the same.

[ciaransweet]

Agree that it will likely break if we switch, but probably an 'expected' break.