Skip to content

release: 0.4.1 with unscripted evaluations and the reviewed video playbook - #10

Merged
phense merged 8 commits into
mainfrom
feat/natural-evals
Nov 13, 2026
Merged

phense merged 8 commits into
mainfrom
feat/natural-evals

Conversation

@phense

@phense phense commented Nov 13, 2026

Copy link
Copy Markdown
Owner

Summary

  • Unscripted routing cases. Nine plain German requests under evals/natural/, run with scripts/run_natural_evals.py from Method Core 64d9a02: no skill name, no routing instruction, both plugins loaded.
  • Video production playbook (WM-001). Three independent content reviews, 43 findings (1 Critical, 20 Important, 22 Minor), all applied; record in docs/specs/WM-001-presentation-video/playbook-review.md. No step rewords a story narration, review gate 1 and the provider mode are checked before paid voicing, delivery keeps the previous file, the correction path is stated. The production skill and the template README no longer name the pause value as a pace lever.
  • Routing corrections. A study vault's instruction file names story-craft-research; the video production skill states that filmed material and short-form clips are outside the video method; explaining-concepts also names short what-is and how-differs questions.
  • FEATURES.md records the working capabilities. Version 0.4.1.

Verification

  • 309 tests pass, 3 skipped (ffmpeg). scripts/validate-plugin, sync-core --check, claude plugin validate --strict pass.
  • Routing harness on Codex after the corrections: marketing-clip-out-of-scope 3 of 3 (before 1 of 3), study-vault-draft-request 3 of 3 (before 0 of 3).
  • Unscripted requests, three runs per case, twelve cases across both plugins: Claude 33 of 36 runs, Codex 35 of 36. The concept question rose from 1 of 3 to 3 of 3 on Claude after the description change.

Limits

  • The corrected playbook has not been rehearsed end to end and its last edits are not reviewed again; no person has done the listening check. WM-001 stays open for that.
  • Three runs per case show a tendency, not a rate. Produced text is not read. Three expectations or scoring rules were changed after seeing results; each is recorded in docs/specs/WM-003-story-family/rehearsal.md.

🤖 Generated with Claude Code

phense and others added 8 commits November 13, 2026 20:12
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Applies the 22 findings of the independent playbook review: no step rewords a
story narration, pace is not adjusted through the pause value, review gate 1
and the provider mode are checked before paid voicing, delivery copies to a
new name and keeps the previous file, the correction path and the inputs are
stated. The production skill and the template README no longer name the pause
value as a pace lever.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…its outer scope

Also records the working capabilities in FEATURES.md.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… (WM-001)

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ffers questions

Unscripted concept question on Claude: 1 of 3 loads before, 3 of 3 after.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ideo playbook

Syncs method-core 64d9a02, records the measured unscripted runs and bumps
the version.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…nd listening check

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@phense
phense merged commit 4287ad2 into main Nov 13, 2026
2 checks passed
@phense
phense deleted the feat/natural-evals branch November 13, 2026 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant