Repository navigation
fix: choose a heteronym's reading from the words around it - #33
Conversation
重 in 「教学任务很重」 was annotated chóng, "to repeat", where the sentence plainly means "very heavy" — zhòng. Nothing in the dictionary separates the two. ReadingComposer ranks a character's readings by preferring lowercase, then a neutral tone, then whichever reading the dictionary says most about; 重 has neither a capital nor a neutral tone, and CC-CEDICT gives chóng five senses to zhòng's three. The sense count is a frequency proxy, and it is wrong for exactly the characters whose readings are different words. No context-free default could have fixed it, because both readings are right somewhere: 重 is chóng in 重复 and zhòng in 很重. So the composer now takes a ReadingContext, and HeteronymRules answers for the cases a closed-class neighbour settles grammatically — a degree adverb modifies a stative predicate, so 很 before 重 rules chóng out rather than merely making it less likely. Negators are a separate, shorter table: 不 admits verbs as readily as adjectives, and earns its place only where both readings it allows point the same way. The table holds only the five characters where context overturns the context-free answer. 好, 难, 累, 大 and 少 are already ranked correctly and are asserted to stay that way, so a change to the generic ranking fails a test rather than quietly needing more rules. Readings are now keyed on the word and its context rather than on the word alone — a heteronym has no single answer per book — and one core function classifies neighbours for both the model, which decides what to fetch, and the paragraph view, which decides what to look up, so the two cannot disagree. The detail panel's summary follows the reading shown above the word, which it did not: after the rule chose zhòng the panel still read "to repeat" beside it. Verified on the running app over 「我在一个山村学校当老师，教学任务很重。他重复 了三次。任务不重。」 — 很重 and 不重 both zhòng, 重复 still chóngfù. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
PR Summary by QodoChoose heteronym readings from grammatical context
AI Description
Diagram
High-Level Assessment
Files changed (16)
|
Code Review by Qodo
1.
|
![qodo-free-for-open-source-projects[bot]](https://avatars.githubusercontent.com/in/1068000?s=60&v=4){kind=small}
价 in 「她叫我给个价」 was annotated jiè, "great, middleman", where the sentence means "name me a price" — jià. Unlike 重, this is not a context problem. jiè is not a reading of modern Chinese, so no neighbour needs to rule it out; the ranking simply believes it. CC-CEDICT documents the archaic sense in four glosses against three for jià, and writes it with a neutral tone besides, so both the neutral-tone preference and the sense count point at it. Removing either would not help. No automatic signal fixes it either. Compound attestation looks decisive — 161 headwords spell 价 jià and none spell it jiè — but applied as a ranking it flips every sentence-final particle, since 吧, 呢, 啊 and 吗 are absent from compounds too, for reasons that have nothing to do with being archaic. So the corrections are a generated table, not a ranking and not a curated list. Scripts/heteronym-defaults.py applies one criterion to the pinned snapshot — a reading used by no multi-character headword beside one used by at least twenty, excluding readings glossed as grammatical — and prints the table verbatim. It yields five characters: 价, 椎, 炮, 络, 膏. Each is a dead reading outranking a living one, and the particles are left alone. The correction sits below the context rules and above the ranking: a neighbour is a reason, a default is the absence of one. The two tables are disjoint today, so that order decides nothing yet, and a test fails the moment they overlap rather than asserting an accident. Applied at lookup rather than baked into the per-character table, so an already-imported container does not need rebuilding — and applied on the composed path too, or a word assembled from its characters would spell 价 jiè while the same character tapped on its own read jià. Verified on the running app: 听力64 「她叫我给个价」 now reads jià, panel "price". Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Second defect, same area, different cause — pushed as Reported: 听力64, 「卖主是位六十多岁的老太太，她叫我给个价」 — 价 read Not a context problem. The neutral-tone preference fires and the sense count agrees, so dropping either rule changes nothing. No automatic signal fixes it. Compound attestation looks decisive — 161 headwords spell 价 So: a generated table, not a ranking and not a curated list.
The correction sits below the context rules and above the ranking — a neighbour is a reason, a default is the absence of one. The two tables are disjoint today, so that order decides nothing yet; rather than assert an accident, a test fails the moment they overlap. Applied at lookup rather than baked into the per-character table, so an already-imported container needs no rebuild, and applied on the composed path too — otherwise a word assembled from its characters spelled 价 Verified on the running app: 听力64 now reads 🤖 Generated with Claude Code |
Review findings on #33. The bundled dictionary settles 差 against the rule: it records both 不差 [bu4 cha1] "to not lack" and 不差 [bu4 cha4] "not bad" as headwords, so a negator decides nothing there. Removed, with a test asserting a negator leaves 差 to the ranking while a degree adverb still settles it — 很差 is only ever "very poor". That asymmetry is the difference between the two classes, so it is now pinned rather than described. The split-compound concern does not reproduce: both shipped segmenters keep 重复 whole, because a compound in the lexicon is what maximum matching matches, and the repair pass only ever merges. But the rule does lean on that, so it is now a test rather than an assumption — 很重复, 不重复 and 这个任务很重复 keep 重复 as one token, no lone 重 reaches the rule, and a character-by-character segmentation is shown being repaired back to 很 | 重复. Two doc comments ended up on the wrong declarations, both because a new function was inserted between a comment and what it documented: contexts(in:) lost its documentation to context(at:in:), and readingsByWord lost the note about 阅兵式 to readings(for:). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
All four review findings addressed in 2 — 差 in the negator table: correct, and the bundled dictionary settles it. CC-CEDICT records both as headwords: So a negator decides nothing for 差. Removed. A degree adverb still settles it — 很差 is only ever "very poor" — and that asymmetry is exactly what separates the two classes, so it is now a test rather than a sentence in a comment. 1 — split compounds: does not reproduce, but the rule did lean on an untested assumption. Both shipped segmenters keep 重复 whole ( 3 and 4 — doc comments on the wrong declarations: correct, and my error. Both from inserting a function between a comment and what it documented: 384 tests. One more mutant planted — 差 put back into the negator table — and killed. 🤖 Generated with Claude Code |
1 participant
The defect
In 「我在一个山村学校当老师，教学任务很重」 the reader annotated 重 as
chóng— "to repeat" — where the sentence means "very heavy",zhòng.The sentence segments to
… | 任务 | 很 | 重, so 重 is a lone token.ReadingComposerranks its readings by preferring lowercase, then a neutral tone, then whichever reading the dictionary says most about. 重 has neither a capital nor a neutral tone, so the sense count decided it — and CC-CEDICT holds:LexiconBuilder.CharacterReadingsranks on the same sense count, so both the whole-word and the per-character paths producedchóng. The sense count is a frequency proxy, and it is wrong for precisely the characters whose readings are different words.ReadingComposeralready named 为, 长, 重 and 相 as the cases it gets wrong.Why it needed context
No context-free default could fix it, because both readings are right somewhere: 重 is
chóngin 重复 andzhòngin 很重. Only the neighbouring words separate them.So
ReadingComposernow takes aReadingContext, andHeteronymRulesanswers for the cases a closed-class neighbour settles grammatically. Closed classes only, deliberately: a rule over an open class needs a tagger, while a rule over a list of two dozen adverbs is a finite fact about the grammar that a reviewer can check.bó.The table holds only the five characters where context overturns the context-free answer — 重, 长, 空, 差, 薄. 好, 难, 累, 大 and 少 are already ranked correctly, and a test asserts they stay that way, so a change to the generic ranking fails there rather than quietly creating the need for more rules.
This is not a part-of-speech tagger and is not a step towards one.
Consequences
ReadingContext.contexts(in:), classifies neighbours for both the model, which decides what to fetch, and the paragraph view, which decides what to look up — so the two cannot disagree. A test pins it against the single-token classifier a tap uses.zhòng, the panel still read "to repeat" beside it, because the summary came from whichever group the dictionary listed first. The expanded sheet leads with the same group and still lists every reading.Verification
Verified on the running macOS app over 「我在一个山村学校当老师，教学任务很重。他重复了三次。任务不重。」:
chóngzhòng, panel "heavy"chóngzhòngchóngfùchóngfù(unchanged)21 new tests; 372 pass. Nine mutants planted against the new code and all nine killed — including three that survived a first attempt and exposed a real gap: the single-token and whole-block classifiers agreed only because no test put a degree adverb behind punctuation.
Known limits
Still a guess, and still marked
.ambiguousso the ruby is drawn faintly and the panel lists every reading. 为 remains wrong (wéiwhere running text usually wantswèi) — it has no closed-class neighbour that settles it, and a headword-count frequency proxy pickswéitoo.🤖 Generated with Claude Code