Repository navigation
fix(conversation): keep error retries under their own threshold - #6236
Draft
yuhengcai1 wants to merge 1 commit into
Draft
yuhengcai1 wants to merge 1 commit into
yuhengcai1 wants to merge 1 commit into
Conversation
Co-authored-by: openhands <openhands@all-hands.dev>
Contributor
|
📁 PR Artifacts Notice This PR contains a |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
HUMAN:
AGENT:
Why
action_observationcurrently countsAgentErrorEventresults as observations. When it is lower thanaction_error, identical failing calls can terminate the conversation before its configured corrective nudge. For example,action_error=5with the default observation threshold stops after four failures without a nudge.Summary
Restrict the action-observation matcher to
ObservationEventresults, keeping tool errors under the existing action-error nudge/stop rule. Parameterize the native run-loop regression across error thresholds 3/5 and observation thresholds 1/4. This is independent of #6235 and based directly on main.Documentation: OpenHands/docs#1008
Issue Number
No existing issue identified for this threshold interaction. This follows the configurable recovery behavior introduced by #4332; it does not change context-window or multi-action detection.
How to Test
On main
ce5716a3c0a955a85bb0cc4dcfaa94d5fefcda50, the native nudge suite with these tests has 3 failed, 3 passed: the low observation threshold cuts off retries after 1 or 4 errors. With the fix, all 18 tests in the command above pass. Each threshold combination emits exactly one nudge after the configured number of failures and stops only after the next failure.The existing successful-observation threshold, user-reset, and recovery tests are negative/regression controls. Pre-commit formatting, lint, typing, and repository checks pass.
Validation uses Python 3.13.17 on macOS arm64, the committed uv lock, and
TestLLMwith the realAgent, failing executor, event construction, andConversation.run()path. No paid inference or real-model benchmark was run.Video/Screenshots
Non-visual change; before/after reproduction results are above.
Design Doc
Small matcher correction; no separate design document.
Type
Notes
Default thresholds, serialized state, and other stuck patterns are unchanged. TypeScript already excludes error results from its action-observation matcher. Other enabled patterns can still terminate a conversation independently.
The HUMAN section remains for the contributor. A maintainer must apply
integration-testfor the agent-behavior evaluation; this contributor account lacks permission to add labels.