Skip to content

fix(skill): auto-generate crystallizer summary/steps instead of rejecting - #2253

Merged
Hun-ger merged 1 commit into
MemTensor:fix-local-plugin-260824from
kiwipaulrob:fix/crystallizer-validator-auto-fix
Aug 26, 2026
Merged

fix(skill): auto-generate crystallizer summary/steps instead of rejecting#2253
Hun-ger merged 1 commit into
MemTensor:fix-local-plugin-260824from
kiwipaulrob:fix/crystallizer-validator-auto-fix

Conversation

@kiwipaulrob

Copy link
Copy Markdown
Contributor

Summary

defaultDraftValidator in core/skill/crystallize.ts throws skill.crystallize.invalid: missing summary / missing steps whenever the LLM returns valid JSON that omits those fields. With deepseek-v4-flash this happens routinely, flooding bridge logs with 10+ ERROR skill.crystallize.failed messages per minute and stalling the crystallizer queue (which in turn delays other RPCs like memory.search waiting on the queue to drain).

This PR makes the validator repair such drafts instead of rejecting them:

  • Missing summary → auto-generated from the richest available field, in priority order: first step body → first step title → displayTitlename → static placeholder "skill procedure". Capped at 200 chars. Never throws for a missing/empty summary.
  • Missing/empty steps → a single "Execute the fix" step generated from the summary (title + body, capped at 2000 chars). Only throws when nothing at all can be derived.
  • Missing name → still rejected (skill.crystallize.invalid: missing name). On the LLM path this is unreachable anyway — normaliseDraft() already substitutes skill_<policy_id> — so the check remains purely a guard for direct validator use.

Change

  • core/skill/crystallize.tsdefaultDraftValidator rewritten: strict → lenient repair (uses || not ?? — LLM JSON emits empty strings, and ?? only falls through on null/undefined).
  • tests/unit/skill/crystallize.test.ts — the old "rejects drafts that the validator flags as invalid" test used { steps: [], summary: "" }, which is exactly the case this PR now repairs; converted into an end-to-end regression asserting the draft crystallizes with auto-generated summary + steps.
  • tests/unit/skill/crystallize-validator.test.ts — new focused suite (8 tests): full-draft passthrough, never-throws-for-missing-summary, summary fallback chain (step body → displayTitle → name), 200-char cap, empty-string (||) semantics, single-step auto-generation, missing-name rejection.

Tests

  • npx vitest run tests/unit/skill54 passed (10 files)
  • npx tsc -p tsconfig.json --noEmit → clean (exit 0)
  • The repair logic is the same shape already proven in production (local Hermes CT100 ran this validator for ~2 weeks; crystallizer backlog drained, zero missing summary rejections since).

Related

Type of change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)

How Has This Been Tested?

  • Unit tests (npx vitest run tests/unit/skill — 54 passed)
  • Type-check (tsc --noEmit clean)
  • Production-proven shape (the lenient fallback chain has run on Hermes CT100 since ~1 Aug 2026; skill.crystallize.failed missing-summary floods dropped to zero)

Checklist

  • I have read the CONTRIBUTING guidelines
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • My changes generate no new warnings
  • I have added tests that prove my fix is effective
  • New and existing unit tests pass locally

…ting

defaultDraftValidator threw skill.crystallize.invalid: missing summary / missing steps whenever the LLM returned valid JSON without those fields (observed with deepseek-v4-flash), flooding bridge logs every 5-10s and stalling the crystallizer queue. It now repairs the draft instead:
- missing summary: derived from first step body/title, then displayTitle, then name, then a static placeholder; capped at 200 chars
- missing steps: a single Execute-the-fix step generated from the summary; only throws when nothing at all can be derived
- missing name: still rejected (normaliseDraft already supplies a name fallback on the LLM path, so this only guards direct validator use)

Fixes MemTensor#2143
@Memtensor-AI Memtensor-AI added area:plugin OpenClaw & Hermes status:in-progress Someone or AI is working on it | 人工或 AI 正在处理 labels Aug 15, 2026
@Memtensor-AI

Copy link
Copy Markdown
Collaborator

✅ Automated Test Results: PASSED

All tests passed (15/15 executed). memos_local_plugin/unit: 15/15. Duration: 4s

Branch: fix/crystallizer-validator-auto-fix

@Hun-ger
Hun-ger changed the base branch from main to fix-local-plugin-260824 August 26, 2026 08:27
@Hun-ger
Hun-ger merged commit 7b130ef into MemTensor:fix-local-plugin-260824 Aug 26, 2026
2 of 3 checks passed
syzsunshine219 added a commit that referenced this pull request Aug 28, 2026
## Description

This PR consolidates the local-plugin reliability fixes developed and
validated on `fix-local-plugin-260824`.

### Highlights

- hardens Hermes bridge process detection, PID/status reconciliation,
installer rollback, Windows-native provider behavior, and UTF-8 tool
output;
- makes secret environment fallback explicit and wires first-class
`maxTokens` / `headers` configuration through the L3 and Skill Evolver
clients;
- moves idle-skill archival off the foreground path with bounded, atomic
lifecycle work;
- recovers useful crystallizer drafts when LLM output omits derivable
fields;
- bounds startup-recovery shutdown and adds the newest-trace index with
collision-safe migration handling;
- resolves DeepSeek Harness auxiliary reasoning capability from exact
model metadata, with per-route TTL caching, concurrent lookup
coalescing, adapter-update invalidation, and registration-bound race
fallback.

Contribution history is preserved from #2192, #2209, #2246, #2248,
#2252, #2253, #2256, and #2284. The capability-driven implementation
supersedes the negative probe cache proposed in #2285.

No new runtime dependencies are introduced.

Related Issue (Required): Fixes #2278

Reviewers: @hijzy @whipser030

## Type of change

- [x] Bug fix (non-breaking change which fixes an issue)
- [x] Refactor (does not change functionality, e.g. code style
improvements, linting)
- [x] Documentation update

## How Has This Been Tested?

- [x] Unit Test
- `npm run test:unit -- --reporter=dot`: 179 files passed; 1,549 tests
passed and 2 skipped.
- `npm exec vitest run tests/unit/adapters/deepseek-harness-*.test.ts --
--reporter=dot`: 65/65 passed.
- `python3
apps/memos-local-plugin/tests/python/test_hermes_provider_pipeline.py`:
36/36 passed.
- [x] Test Script Or Test Steps
  - `npm run lint`
  - `npm run build:package`
  - `npm run check:hermes-version`
- Ruff 0.11.8 check and format verification for both changed Python
files.
- Packed commit `d76c78aa` and installed the identical tarball into DSH
`0.1.0-rc.6` `web` profiles on macOS and Windows.
- Verified SHA-256
`61e805f862acbe0873581e22cac4592ab0a9ff4d23d607a5f9fac5df93e80253` on
both hosts.
- Verified unsupported-reasoning routes issue zero rejected `off` probes
across repeated/concurrent calls, supported routes retain `off`, adapter
invalidation refreshes the capability, and both installed profiles start
with healthy Host LLM bridges.
- [x] Pipeline Automated API Test
  - GitHub CI will run after PR creation.

> Note: `make format` could not start because Poetry is not installed on
the validation host. The equivalent Ruff 0.11.8 checks for the changed
Python files passed.

## Checklist

- [x] I have performed a self-review of my own code | 我已自行检查了自己的代码
- [x] I have commented my code in hard-to-understand areas |
我已在难以理解的地方对代码进行了注释
- [x] I have added tests that prove my fix is effective or that my
feature works | 我已添加测试以证明我的修复有效或功能正常
- [x] I have created related documentation issue/PR in
[MemOS-Docs](https://github.com/MemTensor/MemOS-Docs) (if applicable) |
N/A: no external documentation change is required.
- [x] I have linked the issue to this PR (if applicable) | 我已将 issue
链接到此 PR
- [x] I have mentioned the person who will review this PR | 我已提及将审查此 PR
的人

## Reviewer Checklist

- [x] closes #2278
- [x] Made sure Checks passed locally; GitHub CI will run on creation
- [x] Tests have been provided
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:plugin OpenClaw & Hermes status:in-progress Someone or AI is working on it | 人工或 AI 正在处理

Projects

None yet

Development

Successfully merging this pull request may close these issues.

crystallize: defaultDraftValidator throws skill.crystallize.invalid: missing summary when LLM omits summary field

4 participants