Skip to content

fix(sleep): separate optimizer feedback from judge evidence - #270

Draft
diqierjia wants to merge 1 commit into
microsoft:mainfrom
diqierjia:codex/skillopt-feedback-separation
Draft

fix(sleep): separate optimizer feedback from judge evidence#270
diqierjia wants to merge 1 commit into
microsoft:mainfrom
diqierjia:codex/skillopt-feedback-separation

Conversation

@diqierjia

Copy link
Copy Markdown

Summary

Related to #154.

This draft addresses one concrete, reproducible failure mode: raw judge evidence can leak regexes and checker implementation details into SkillOpt-Sleep optimizer prompts. The optimizer may then learn the verifier syntax instead of the user-visible behavior.

Scope

  • Add an optional ReplayResult.optimizer_feedback channel for learning-safe semantic feedback.
  • Keep fail_reason and judge_rationale unchanged as raw audit/debug evidence.
  • Route ordinary reflection, contrastive reflection, and slow update through semantic feedback only.
  • Prefer a check's natural-language description; if it is absent, regex checks use a conservative non-pattern fallback.
  • Fail closed when a regex is copied into its description or when legacy non-rule evidence cannot be safely projected.
  • Preserve check descriptions emitted by the miner.
  • Protect the contributed OpenClaw reflection path from bypassing the shared safety boundary.

Data flow

Raw judge rationale remains available in replay results, evidence logs, and reports for auditability. Optimizer prompts receive only optimizer_feedback (or a recomputed safe projection for legacy results). Rule scoring keeps the existing backward-compatible score_rule_judge() return shape; the new helper carries the additional semantic feedback.

Backward compatibility

optimizer_feedback is optional and defaults to an empty string. Older or manually deserialized ReplayResult objects without that attribute remain usable: rule tasks recompute feedback from the task judge, while non-rule legacy evidence falls back to a generic message rather than exposing raw rationale.

Validation

  • Focused regression suite: 220 passed, 3 skipped.
  • Full suite: 1440 passed, 74 skipped, 2 failed. The two failures are the existing Windows symlink tests, both blocked by WinError 1314 (the process lacks the privilege required to create symlinks). No tests were changed to hide this environment limitation.
  • python -m compileall -q skillopt skillopt_sleep plugins scripts tests: passed.
  • git diff --check: passed.
  • Ruff passes for the changed OpenClaw backend file. The repository-wide Ruff baseline still reports pre-existing findings outside this focused change.

Non-goals

This is intentionally not a solution for the full intent-level mining or proxy-alignment problem described in #154. It is a narrow containment fix for judge-evidence leakage while preserving existing scoring and audit behavior.

@diqierjia
diqierjia force-pushed the codex/skillopt-feedback-separation branch from d5f9869 to b00456b Compare September 4, 2026 08:36
@diqierjia
diqierjia force-pushed the codex/skillopt-feedback-separation branch from b00456b to 53e3fac Compare September 4, 2026 08:41
@diqierjia

Copy link
Copy Markdown
Author

@microsoft-github-policy-service agree

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant