Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 10 additions & 0 deletions docs/design/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -46,6 +46,13 @@ the 2026-08-16 entries below. The survivor is the degenerate bare
"van Mesnil Juan" gives given="van", middle="Mesnil",
family="Juan". given="de" is the reading the vocabulary exists to forbid, and the fold is what prevents it. The all-orders agreement in P1 is deliberate and is W4's shape: a wholly-hangul name reads family="김" under every declared order because the script carries a signal the order does not override, and a leading never-given particle is the Latin-script analogue. decisions.md#O4 already draws the line — "Words no vocabulary has claimed read by position" — so name_order governs the unclaimed remainder, which is most inputs.

- 2026-08-17 — SUPERSEDES the order-independence half of the 2026-08-16 order-precedence keystone entry above (#364, #365, #368). That entry says "no name_order moves that stopping point"; it may. Declaring FAMILY_FIRST or FAMILY_FIRST_GIVEN_LAST is precisely an assertion that what follows the family is NOT more surname, and a particle run's stopping point is exactly the question of where surname material ends — so the declaration is evidence about it, not merely about which slot the result lands in. The grouping/assignment split survives everywhere else; what is narrowed is the claim that the stopping point is one of the things order cannot touch.
Concretely: under GIVEN_FIRST a leading particle run keeps joining to the end of the name, which makes "de Mesnil Juan" read exactly like "pennie von bergen wessels" — the same shape, and this session decided the greedy reading is right for that one. Under a family-first order the run stops after its first non-particle name word, because the caller has said the rest is given or middle.
Accepted, and it is the cost that decided #364 three separate ways before this: "de la Vega Juan" under the DEFAULT order stays wholly a surname. A caller who means family "de la Vega" plus given "Juan" writes the comma. The mixed-language shapes are what make the greedy reading look wrong, and the parser cannot see language.
Why the question kept thrashing, worth recording so a fourth attempt does not start from scratch: "how much does a leading particle run take" is a LANGUAGE judgement being forced through a POSITION heuristic, in a parser that has correctly refused to detect language (decisions.md#O4, rules.md's Not-in-scope). Every argument in the thread — `de`/`do` as Vietnamese surnames, `von` as German, `dos` as Portuguese — is really about which tradition the name belongs to. The order declaration is the one place a CALLER supplies that information instead of the parser guessing, which is why keying on it is better than any position rule we tried.
Implementation consequence: the stopping point may now read name_order, which the superseded sentence forbade. Whether that lives in grouping (order-aware) or as a split in assignment is open — PR #391 showed assignment cannot split an existing piece today, so choosing assignment means giving it token-level slicing.
Supersedes nothing else: #368's reversal, the trailing-orphan rule P6, and the claim that a leading never-given particle takes the FAMILY rather than a given name all stand.

Declined:

- A strict xfail asserting "de Mesnil" → family under FAMILY_FIRST (#359 review) — #359 deliberately left those semantics open, and a strict xfail decides the question by the back door.
Expand Down Expand Up @@ -190,6 +197,7 @@ Open: [#348](https://github.com/derek73/python-nameparser/issues/348) applying C
### P3 — connectives

- 2026-07-30 #267 — the four-word single-letter asymmetry: a bare Latin capital connective is vetoed as an initial while a Cyrillic capital joins. #267's closure ("v2.0 behaves this way by default … verify it was the right call") showed only the Cyrillic half; the Latin-capital veto was never separately adjudicated, which rules.md#P3 records as an Accepted consequence.
- 2026-08-17 — a conjunction-joined run counts as ONE name word in every rule that counts them. Raised while bounding the leading particle run (#395): a run that stops "after the first name word" would otherwise be able to stop INSIDE a part P3 built, splitting "de la Vega y Santos Juan" between "Vega" and "Santos". This is not new behavior — H1 already counts this way, measured: "Mr. Jack and Jill" gives family="Jack and Jill" while "Mr. Jack Jill" gives given="Jack", family="Jill". What is new is that rules.md says so, which is what binds #395's implementation. Recorded because the first draft of the clause asserted that no rule could show it today and shipped an Accepted note to that effect; the shipped H1 pair was simply never run. Detector failure of the usual shape — reasoning about which rules count instead of parsing a name and looking.

Open:
[#383](https://github.com/derek73/python-nameparser/issues/383)
Expand Down Expand Up @@ -536,3 +544,5 @@ Promoted 2026-08-15 from session memory (Derek's 2026-07-30 ask; promotion appro
- (A) Parking lot from the 2026-07-06 config-model discussion, 3.0-shaped by design: scoped config override via contextvars (`with nameparser.config.use(c):`, the decimal.localcontext pattern — fixes test pollution and config wars while keeping the shared default), and a requests-Session-style Parser(constants) API (rejected then as a full rewrite; the shared CONSTANTS survives 2.x because it is the only config channel that reaches parses inside code users don't own — #262's contract framing).
- (B) The 1.3.0-era legacy-pickle property-key skip (suffixes_prefixes_titles in old blobs) rides in the shim's __setstate__ and dies with it.
- (B) The FAMILY_COMMA doctrine (rule W3): inherited from v1's lastname-comma but correct on its own terms — an explicit comma is stronger evidence than script.
- (A) 2026-08-17 — PARTICLE VOCABULARY AS LOCALE PACKS. Raised while reversing the order-independence decision above: every argument in the leading-particle thread is a language judgement wearing a position heuristic. `de` and `do` lead surnames in Vietnamese, `von` is German, `dos` Portuguese, `das` both Portuguese and a borne Bengali surname — and the per-word comments in nameparser/config/particles.py cite exactly these traditions as their justification. Locale packs already exist (ru, tr_az, zh, ja) and already carry vocabulary and rotations, so particles are the obvious next tenant: a caller who knows the tradition could opt into a tuned set instead of the parser splitting the difference globally. Not 2.x work — it needs the packs to carry vocabulary OVERRIDES rather than additions, and it would give C-i's positional qualifier a per-locale answer rather than one global one.
- (A) 2026-08-18 — LOCALE CONTROL OF P3's SINGLE-LETTER CARVE-OUT, and the sharper requirement it puts on the entry above: a pack would have to carry per-word POLICY, not just membership. Which single letters a tradition wants joined differs by language — Spanish writes "y" and would want "e" read as an initial, Portuguese wants the reverse, Dutch would want every single letter read as an initial. One global rule serves all three today: a single ALPHABETIC character in a name of fewer than four pieces stays a name word, and a bare Latin capital never joins (#383). It is v1's Google Code issue 11 fix ("john e smith") generalized to a vocabulary v1 did not have — the set now holds six single-letter conjunctions (y, e, и, і, й, و), and the carve-out reaches all of them because it counts characters, while the capital veto reaches only the Latin ones because it tests a Latin shape. That asymmetry is the visible seam and it is why #383 is open. A pack that could only ADD or REPLACE vocabulary could not express any of it; the join threshold and the initial veto would have to be per-word, per-pack values. Measured while documenting P3: "&" is single-character but not a letter, so it joins at any length ("Juan & Garcia" is one part while "Juan y Garcia" is three).
23 changes: 20 additions & 3 deletions docs/design/rules.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Every example line is EXECUTABLE. The grammar (its executable definition is `tes
"INPUT" [annotation] → field="value" [· boundary]
[deviates: #N (today: field="value")]

An `annotation` names a policy, locale (`[ru]`), or extras gate (`[ja+segmenter]`) in the registry beside the test. `· boundary` marks the non-firing example every rule must carry — or the rule declares `no-boundary: <reason>` instead, so skipping the boundary is a recorded decision. `deviates:` states the INTENDED output on the example line while the marker records TODAY's output and the tracking issue; the runner asserts today's output strictly, so a parser change that closes the gap fails the suite until the marker is removed in the same PR. `grep deviates:` on this file is the deviation backlog (deviations from statable rules — coverage gaps are a separate, larger category no grep can see, and contested vocabulary memberships a third, tracked as Open blocks keyed to the vocabulary set in decisions.md).
An `annotation` names a policy, locale (`[ru]`), or extras gate (`[ja+segmenter]`) in the registry beside the test. `· boundary` marks the non-firing example every rule must carry: an input shaped like the rule's subject where its effect does NOT occur. That is usually the rule's OWN stated exception — H1's given-name title, P3's single-letter carve-out — not an input the rule never reaches, so the exception is executable rather than merely asserted. Or the rule declares `no-boundary: <reason>` instead, so skipping the boundary is a recorded decision. `deviates:` states the INTENDED output on the example line while the marker records TODAY's output and the tracking issue; the runner asserts today's output strictly, so a parser change that closes the gap fails the suite until the marker is removed in the same PR. `grep deviates:` on this file is the deviation backlog (deviations from statable rules — coverage gaps are a separate, larger category no grep can see, and contested vocabulary memberships a third, tracked as Open blocks keyed to the vocabulary set in decisions.md).

## Not in scope

Expand Down Expand Up @@ -150,16 +150,33 @@ P3. Rationale: connective words ("y", "of the") bind name words into
connective runs included — except a single-letter connective in
a three-word name, which stays a name word, and a single-letter
connective written as a bare Latin capital, which reads as an
initial and never joins.
initial and never joins. The joined part is ONE name word
wherever another rule counts them, so a rule taking "one name
word" takes the whole join and never half of it.
"Juan y Eva Garcia" → given="Juan y Eva"
"Jose E Maria Santos" → middle="E Maria"
"Juan y Garcia" → middle="y" · boundary
"Juan and Garcia" → given="Juan and Garcia"
"Juan & Garcia" → given="Juan & Garcia"
"Mr. Jack and Jill" → family="Jack and Jill"
"Mr. Jack Jill" → given="Jack"
Both exceptions are about the written FORM, not the word: the
three-word carve-out counts letters, so a symbol connective joins
at any length, and it reaches every single-letter connective the
vocabulary holds — Cyrillic и/і/й and Arabic و as well as y and
e. Which single letters a tradition actually wants joined differs
by language, and no locale gets its own answer today.
Accepted: the initial veto is a LATIN shape — a Cyrillic
capital joins ("И".isupper() is true, so this is not a
Unicode-uppercase rule); #267's closure blessed the Cyrillic
side, and whether the Latin-capital half should stand is #383.
"Хосе И Мария Сантос" → given="Хосе И Мария"
history: decisions.md#P3 · implemented: nameparser/_pipeline/_group.py
H1 is the counting rule that shows the one-word clause today: a
title plus the join reads the whole join as the family, where the
same two words unjoined are two name words and H1 does not fire.
P1's leading run becomes the second once #395 lands — its run
must take the "Vega y Santos" join whole or stop before it.
history: decisions.md#P3 · interacts: H1, P1 · implemented: nameparser/_pipeline/_group.py

P4. Rationale: a particle links forward from inside a name; at the
very front there is no name yet to be inside.
Expand Down
10 changes: 6 additions & 4 deletions nameparser/_pipeline/_group.py
Original file line number Diff line number Diff line change
Expand Up @@ -187,10 +187,12 @@ def merge(lo: int, hi: int, add: Set[str] = frozenset(),
merge(k, k + 2, add={"conjunction"})
else:
k += 1
# each conjunction joins its neighbors (v1's Google Code issue 11
# carve-out, the "john e smith" bug:
# a single-letter alphabetic conjunction in a short name is more
# likely an initial)
# each conjunction joins its neighbors, rules.md#P3: "except a
# single-letter connective in a three-word name, which stays a
# name word" (v1's Google Code issue 11 carve-out, the
# "john e smith" bug). The threshold reads the ROOTNAME count,
# so a conjunction that is also suffix vocabulary raises the
# bar for itself -- #397 measures that on "i".
k = 0
while k < len(pieces):
if not conj(k):
Expand Down