Skip to content

fix(tools): persist tool output reference labels and keep keys unique across runs - #629

Open
Shabonasar wants to merge 1 commit into
LibreChat-AI:mainfrom
Shabonasar:fix/stable-tool-output-ref-labels
Open

Shabonasar wants to merge 1 commit into
LibreChat-AI:mainfrom
Shabonasar:fix/stable-tool-output-ref-labels

Conversation

@Shabonasar

@Shabonasar Shabonasar commented Oct 20, 2026 •

Copy link
Copy Markdown

The [ref: tool<i>turn<n>] label is now the stored name of a tool output: it is saved with the result and restored with the history, and turn numbers continue across runs. Before, the label was shown only while the key was in the current run's registry and every run restarted at turn0, so:

  • the restored history lost the labels: the first request of each later turn missed the prompt cache and Claude dropped thinking blocks bound to the old prefix;
  • keys repeated between turns: {{tool0turn0}} from an old turn silently resolved to the current turn's output.

Fixes #627

live:     ToolNode key tool0turn7 -> ON_RUN_STEP_COMPLETED.tool_call { output, ref, unresolved_refs? } -> host stores it
restore:  formatAgentMessages: ref -> ToolMessage.additional_kwargs._refKey
          Run: turn counter = highest restored turn + 1   (keys never repeat)
request:  annotateMessagesForLLM labels by _refKey, no registry lookup, after truncation (byte-equal to live)
  • An old key is not in the registry, so it stays literal with the existing [unresolved refs: ...] warning.
  • Handoff agents share one registry (one seed); subagents have their own run and history.
  • Tests that expected "no label when the key is missing from the registry" now expect the label.
  • LibreChat needs no change: the fields ride in the stored tool_call part.
  • Not changed: a placeholder in the same parallel batch as the call it names stays literal without a warning.

Measurements

Real Anthropic API through LibreChat, 6 turns, adaptive thinking, code interpreter running code, chained bash_tool calls, MCP tool, restart after turn 3, turn 5 references a key from turn 1. "series" = the LibreChat series plus agents #613, #614, #615, #619 (LibreChat-AI/LibreChat#17102); dev = LibreChat dev with agents main. This PR has no dependency on the series and was added on top of each. "before" used a 1h cache TTL, "after" 5m.

model LibreChat agents cache hit misses dropped thinking blocks 400
Haiku 5.5 series #17102 before 87.9% 4 28 0
Haiku 5.5 series #17102 this PR 92.9% 0 0 (2nd run: 1, from #618) 0
Sonnet 5.5 series #17102 before 86.1-88.4% 3 10 0
Sonnet 5.5 series #17102 this PR 90.8% 0 0 0
Haiku 5.5 dev before 88.6% 5 0 0
Haiku 5.5 dev this PR 87.5% 5 0 0

With the series, keys run tool0turn0...tool0turn8 without repeats, and the turn-5 probe returns [unresolved refs: tool0turn0] instead of a wrong output. On dev the labels are now stable too, but its misses come from other restore differences fixed by the series (LibreChat-AI/LibreChat#17102).

Testing

  • Full suite: 6568 passed; the 16 failures also fail without this change (3 llm specs need API keys, 2 handoff/checkpoint specs).
  • New ToolNode.persistedReferences.test.ts (9): fields on direct, early and eager completions; none when the feature is off; aggregator -> persistence -> restore gives the live wire content; turn seeding; old key stays unresolved.
  • tsc --noEmit, eslint, check:circular-deps clean.

Type of change

  • Bug fix

Checklist

  • I reviewed my own changes
  • Relevant tests have been added or updated
  • Existing relevant tests pass
  • The change does not introduce new warnings or errors

… across runs

[ref: toolNturnM] labels were added only when the key was live in the current
run's registry, and every run restarted its turn counter at 0. A restored
history therefore had no labels (the first labelled result of each later turn
changed the prefix and missed the provider cache), and keys repeated between
turns, so {{tool0turn0}} from an old turn silently resolved to the current one.

- ON_RUN_STEP_COMPLETED tool_call now carries `ref` and `unresolved_refs`
  (direct, early, eager and batch completions); the content aggregator keeps them
- formatAgentMessages restores them as `_refKey` / `_unresolvedRefs`
- annotateMessagesForLLM labels by `_refKey` without a registry lookup
- Run seeds the registry turn counter past the highest restored key, so a
  placeholder naming an older turn is reported as unresolved

Fixes LibreChat-AI#627

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool output reference labels missing from restored history: cache misses and wrong {{toolNturnM}} substitution

1 participant