[click0]

Third bug hunt. It had two parts:

• a fuzzer for tables and CSV, numbered lists, Latin look-alike letters, double first names and documents with dozens of people, plus large-document runs of up to 8000 lines;
• an audit of the extras API, the compatibility module and the old mapping formats.

All entries are in CHANGELOG.md under [Unreleased].

Data left unmasked

• CSV from Excel: a ; glued to the next word now separates cells like |. Before, the patronymic in Петренко Іван Петрович;сержант;1234567890 stayed open. A comma glued to the next word reads as , .
• Latin look-alike letters inside names (Ярoслав): the mask could be the real name, and a nearby surname stayed open. Names are now analysed with such letters read as Cyrillic.
• Rank forms: ranks with запасу/резерву and dative ranks in -ові (майорові, солдатові) are now recognised. The -ові form is a new rank form, rank_data.RANK_DATIVE_OVI, and the mask also has it.
• A multi-word rank right after another rank (… капітаном / СТАРШИЙ ЛЕЙТЕНАНТ … on joined lines) had its first word masked as a surname.

Unmasking did not restore the original

• Occurrence numbering: occurrences are now numbered regardless of the mask's letter case. Masks that differ only in case (ГОРДІЇВНИ / Гордіївни) left an occurrence unrestored. New mappings carry "instance_case": "insensitive"; older ones are read as before.
• Letter case of double names: the case is now taken part by part from the text. Before, ДОРОШЕНКО-ҐУДЗЬ came back in capitals in ordinary lines, and Анна-Марія came back as Анна-марія.
• Rank masks in long orders: a rank's mask could equal an open rank. Now the nearest free rank of the whole hierarchy is taken, and if none is free the rank is left as it is. Before, documents of 500, 2000 and 8000 lines with 300 people failed the round trip; all three now pass.
• Old mapping formats:
  • v2.0 entries without instances now restore every occurrence of their mask; before, they restored nothing.
  • v2.0 ranks are now restored.
  • A mapping without version is classified by structure.
  • v1 mappings are restored in one whole-word pass; before, they re-replaced restored values and replaced inside longer words.

Masks

• One synthetic stem per surname in a document: Коваль/Коваля/Ковалем → Ковар/Коваря/Коварем, as CLAUDE.md promises. Before, each case form got a different stem.
• Surname masks use the ASCII apostrophe, so documents in cp1251 save. Before, a mask with ʼ made the save fail.

API and command line

• data_masking: the general flags (MASK_NAMES, MASK_DATES, MASK_ORDERS) now also set the flags derived from them, and apply_filter_to_globals on the module's dict now works.
• tools: name and patronymic gender is detected from the value, not assumed male.
• generate_password now guarantees every character class.
• MappingChain / ChainUnmasker:
  • the forward mapping is no longer one pass off;
  • chains whose passes have no "pass" number now work;
  • the passes setter keeps the pass count;
  • from_dict now copies its input.
• An empty --only or --exclude list is an error (exit code 2).

Tests

• tests/test_fuzz_findings3.py: 40 tests; it does not import on main (new API).
• tests/test_api_findings.py: 35 tests; 33 fail on the previous commit.
• Two existing tests were updated:
  • test_rank_data: a new case name;
  • test_unmask: a v2.0 entry without instances now restores.
• Full suite: 1807 passed, 1 skipped. mypy and flake8 are clean.
• Fuzzing after the fixes, about 7500 documents:
  • round trip: 0 failures;
  • --re-mask chains: 0 failures in 250.
• The only leaks the fuzzers still report are two-word names split across lines, a known limitation.

---

Generated by Claude Code