docs: wake-word plan rev 3 — the M4 review, the two decisions, and the M4 section rewritten - #47
Merged
Merged
Conversation
…pointer docs/wake-word-m4-review.md reviews rev 2's "M4 — Corpus, splits and dataset builder" section before it is built, in the rev-1 audit's format: eight adversarial lenses plus an external-facts checker, a skeptic per finding refuting against the checkouts, a completeness critic and a cross-lens judge (104 agents; 93 findings raised, 0 refuted, 15 discounted by the judge). Verdict: the skeleton is right — speaker-disjoint splits keyed across MSWC and Common Voice, one named never-trained-on eval negative set as the FA/hour denominator, manifest plus out-of-git store, dataset card, consented hold-out — but fifteen root issues must be settled before building, one critical: the recorded hold-out is human voice data going into a public MIT repository with consent scope, audio licence, storage form and size undecided, and the RIR fixture generator the plan cites stamps MIT on every fixture it emits (make_rir_fixtures.py:108-109). Also: DspTap's numpy reference is hard-wired to one geometry, so "import, don't restate" cannot serve a tuned parameter; the rebuild pass has no comparison target and no builder fixture or CI job; the Pico-microphone hold-out path has no capture path before M7; eval hours are never sized (3.0/H bounds the rate at zero events); no TTS negatives; music absent from training but targeted in M6; the endpoint undefined. Two facts moved under the plan and were verified on the live pages: the maintained Piper is GPL-3.0 (rhasspy/piper archived 2025-10-06, espeak-ng embedded in OHF-Voice/piper1-gpl), and Common Voice is now account-gated on Mozilla Data Collective with a no-re-hosting clause on top of CC0, so no Common Voice audio may enter any committed fixture or toy corpus. The review lists what survives, the discounted findings, nineteen ordered amendments for a rev 3 of the section, and a staging M4a (builder, contracts, Speech Commands, toy CI fixture) -> M4b (full corpus) -> M4c (hold-out, host path only), with the >= 10-talker / >= 200-utterance target enforced at M6. Formatted version linked from the document. HANDOFF's wake-word entry points at the review and names the two decisions that precede rev 3. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ZkpA4PWuaan5ek2CL2DDm
Taken 8 September 2026, ahead of rev 3 of the plan's M4 section: 1. The hold-out audio stays out of git. FLAC in the off-git store (or a private release asset); the repository commits only holdout.json — per utterance sha256, talker pseudonym, microphone path, distance, SNR, phrase and endpoint sample; per talker pseudonym, consent-form version and permitted uses. The M5 harness refuses a hold-out whose hashes do not match the committed rows. Consent covers evaluation by the project, retention, loudspeaker replay at M7, aggregate numbers, pseudonymous handling and a withdrawal path, never public-repo permanence, plus an optional per-talker opt-in for later CC BY 4.0 release attributed to the collection. The M4c pilot is n = 1. 2. Training features are generated through the DspTap C ABI bridge (dsptap_py.LogMel), which takes every log_mel_geometry value and agrees with the numpy reference to 1.5e-14 at the reference geometry. The numpy script stays M1's oracle; parameterizing it is a DspTap follow-up, not an M4a prerequisite. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ZkpA4PWuaan5ek2CL2DDm
Replaces §6 M4 with the staged M4a (builder, contracts, Speech Commands, toy fixture, kws-dataset CI job) → M4b (full corpus, phrase, splits, lock, card) → M4c (hold-out, host path only) design, carrying every amendment of docs/wake-word-m4-review.md §6–§7 and the two decisions of 8 September 2026: the hold-out audio stays out of git (FLAC in the store, holdout.json the only committed record, hash mismatch refused) and training features come from the shipping front end through dsptap_py.LogMel, the numpy reference staying the oracle. Collateral one-line edits keep the rest of the plan consistent: the header, the §4 rows for Speech Commands, Common Voice, MUSAN, Piper and the hold-out, the §5 kws list (provenance block; label form decided in M6), M0's compute line, M1's as-built sentence, M5's hit window and report columns, M6's pass, M7's bench protocol (the Pico path produced by replay), the documentation table, a §8 access-drift paragraph, §9's shipped-hold-out qualifier and the provenance bullets. Every number in the section is arithmetic on stated inputs or a labelled target; nothing is a measurement. HANDOFF's wake-word bullet now records rev 3 as landed and names M4a as next. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016ZkpA4PWuaan5ek2CL2DDm
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this changes
Documentation only. Adds the adversarial review of the wake-word plan's M4 section (
docs/wake-word-m4-review.md), records in HANDOFF the two decisions it needed, and rewrites the plan's M4 section as rev 3 (docs/wake-word-plan.md) — staged M4a → M4b → M4c — with the collateral one-line edits elsewhere in the plan that keep it consistent.Why
Rev 2's M4 could not be built without the builder deciding roughly a dozen things the plan was silent on, one of them irreversible: consented human voice data was heading into a public MIT repository with consent scope, licence, storage form and size undecided. The review (93 findings, 15 root issues) settled what must be decided before recording or building; the two decisions taken on 8 September 2026 — hold-out audio stays out of git with only
holdout.jsoncommitted; training features come from the shipping front end throughdsptap_py.LogMelwith the numpy reference as the oracle — shape several of rev 3's paragraphs. Two external facts also moved under the plan: the maintained Piper is GPL-3 (espeak-ng embedded) and Common Voice is account-gated on Mozilla Data Collective with a no-re-hosting clause.Verification
96b4572) and DspTap (58210ae); the two external facts with the largest consequences re-fetched by hand (Mozilla Data Collective Data Consumer terms; therhasspy/piperarchive notice).Notes for the reviewer
eval-ttsshare, the ids in the lock) are listed in the M4 section's paragraphs and can be flipped one paragraph at a time.🤖 Generated with Claude Code
https://claude.ai/code/session_016ZkpA4PWuaan5ek2CL2DDm