Skip to content

fix(conversation): keep error retries under their own threshold - #6236

Draft
yuhengcai1 wants to merge 1 commit into
OpenHands:mainfrom
yuhengcai1:codex/stuck-detector-error-threshold
Draft

yuhengcai1 wants to merge 1 commit into
OpenHands:mainfrom
yuhengcai1:codex/stuck-detector-error-threshold

Conversation

@yuhengcai1

Copy link
Copy Markdown

HUMAN:


AGENT:

Why

action_observation currently counts AgentErrorEvent results as observations. When it is lower than action_error, identical failing calls can terminate the conversation before its configured corrective nudge. For example, action_error=5 with the default observation threshold stops after four failures without a nudge.

Summary

Restrict the action-observation matcher to ObservationEvent results, keeping tool errors under the existing action-error nudge/stop rule. Parameterize the native run-loop regression across error thresholds 3/5 and observation thresholds 1/4. This is independent of #6235 and based directly on main.

Documentation: OpenHands/docs#1008

Issue Number

No existing issue identified for this threshold interaction. This follows the configurable recovery behavior introduced by #4332; it does not change context-window or multi-action detection.

How to Test

OH_PERSISTENCE_DIR=/tmp/openhands-stuck-check LITELLM_LOCAL_MODEL_COST_MAP=True \
  uv run --frozen pytest tests/cross/test_stuck_detector.py \
  tests/cross/test_stuck_detector_config.py \
  tests/sdk/conversation/local/test_stuck_detector_nudge.py -q

On main ce5716a3c0a955a85bb0cc4dcfaa94d5fefcda50, the native nudge suite with these tests has 3 failed, 3 passed: the low observation threshold cuts off retries after 1 or 4 errors. With the fix, all 18 tests in the command above pass. Each threshold combination emits exactly one nudge after the configured number of failures and stops only after the next failure.

The existing successful-observation threshold, user-reset, and recovery tests are negative/regression controls. Pre-commit formatting, lint, typing, and repository checks pass.

Validation uses Python 3.13.17 on macOS arm64, the committed uv lock, and TestLLM with the real Agent, failing executor, event construction, and Conversation.run() path. No paid inference or real-model benchmark was run.

Video/Screenshots

Non-visual change; before/after reproduction results are above.

Design Doc

Small matcher correction; no separate design document.

Type

  • Bug fix
  • Feature
  • Refactor
  • Breaking change
  • Docs / chore

Notes

Default thresholds, serialized state, and other stuck patterns are unchanged. TypeScript already excludes error results from its action-observation matcher. Other enabled patterns can still terminate a conversation independently.

The HUMAN section remains for the contributor. A maintainer must apply integration-test for the agent-behavior evaluation; this contributor account lacks permission to add labels.

Co-authored-by: openhands <openhands@all-hands.dev>
@github-actions

Copy link
Copy Markdown
Contributor

📁 PR Artifacts Notice

This PR contains a .pr/ directory with temporary PR-specific documents. The directory will be automatically removed after this PR merges to main through a cleanup PR.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant