Skip to content

Fix leaks and round-trip failures found by fuzzing - #89

Merged
click0 merged 1 commit into
mainfrom
claude/refactor-data-masking-lIcWN
Nov 8, 2026
Merged

click0 merged 1 commit into
mainfrom
claude/refactor-data-masking-lIcWN

Conversation

@click0

@click0 click0 commented Nov 8, 2026

Copy link
Copy Markdown
Owner

A randomized fuzzer (documents with ranks, names in every case and letter case, numbers, dates, non-breaking spaces, tabs, CRLF and hard line wraps; round trip, leak and over-masking checks) found the problems below. All of them except the hang were already on main. Each one has a test in tests/test_fuzz_findings.py: 56 of its tests fail on main, and the rest are guards against over-masking.

Names left unmasked

  • A full name before a rank that has more words after it was not masked at all, e.g. Петренко Іван Іванович доповів капітану про виконання. The parser looked for name anchors only after the first rank in the line.
  • A line break inside a name left part of it open (Петренка\nІвана Івановича, Петренка Івана\nІвановича, Шановний Віталію\nРомановичу!). When a name with a patronymic, or a rank and its name, crosses the break, the two lines are now parsed together, and the break is put back after masking.
  • Initials and a surname on adjacent lines. In a run like Ю.А. Зінченко Н.І. Савчук, only the middle pair used to be masked. The masker now picks the set of matches that covers the most names.
  • Fixed for initials: double surnames in capitals (ПЕТРЕНКО-КОВАЛЬ І.М.) and surnames with an apostrophe (ПИЛИП'ЮК О.І.).
  • Fixed in names: a capitals heading before a name (ВИТЯГ З НАКАЗУ Коваль …), an adjective surname in the genitive (Рапорт Білого Ярослава Миколайовича), and the vocative Любове.

Unmasking did not restore the original

  • A rank masked as генерал-майор медичної служби was not restored. Rank boundaries in the unmasker now exclude a hyphen.
  • A rank with a non-breaking space or tab between its words lost the separator when its mask had a different number of words. The mask now keeps the word count.
  • A rank broken across lines came back with a space instead of the line break.
  • An open rank … медичної служби (one without a name next to it) could become the mask of another rank.
  • In Мороз Ольги Богданівни Л.А. Мороз, the patronymic before the initials was masked as a surname and then masked again.
  • Masks inside one line were recorded in parse order, not text order, so Іване and Івану (both masked as Ігорю) were restored the wrong way round.
  • In a brigade name broken across lines, the end of the name disappeared from the masked text.

Masks

  • Naval ranks now get naval masks (капітан 1-го рангу used to become майор).
  • Білої now gets the same synthetic stem as Біла and Білій.

Checks

  • pytest: 1620 passed. mypy: 0 errors. flake8: clean.
  • Fuzzer, 4000 documents over 8 seeds: no round-trip failures, no leaks, no timeouts. The only flags are cases where a mask happens to equal another person's real name, which is a documented limitation.
  • CLI round trips on both example files pass for plain, --encrypt and --re-mask 2.
  • Against main, the only mask change in input_example.txt is a naval rank. In the generated documents, the changes are the fixes listed above.

Changes are under ## [Unreleased] in CHANGELOG.md. The version is not bumped, per the release-time bump rule.


Generated by Claude Code

A randomized round-trip and leak fuzzer (documents with ranks, names in
all cases and letter cases, numbers, dates, non-breaking spaces, tabs,
CRLF and hard line wraps) found these; each has a test in
tests/test_fuzz_findings.py.

Names left unmasked:
- A full name before a rank followed by more words ("Петренко Іван
  Іванович доповів капітану про виконання") was not masked: the parser
  took name anchors only after the first rank of the line.
- A line break inside a name left part of it open. Lines are joined for
  parsing when a name with a patronymic, or a rank and its name, crosses
  the break; the break comes back after masking.
- Initials and a surname on adjacent lines; runs of initials and
  surnames ("Ю.А. Зінченко Н.І. Савчук"), now resolved by taking the set
  of matches that covers the most names.
- Double surnames in capitals and surnames with an apostrophe before or
  after initials; a heading in capitals before a name; adjective
  surnames in oblique cases; the vocative "Любове".

Unmasking did not restore the original:
- "генерал-майор медичної служби" as a mask (rank boundaries now exclude
  a hyphen).
- Non-breaking spaces or tabs inside a rank whose mask had a different
  number of words; ranks broken across lines.
- An open rank "... медичної служби" used as the mask of another rank.
- A patronymic before initials masked as a surname, then masked again.
- Two forms with the same mask in one line restored the wrong way round:
  masks in a line are now computed in text order.
- A brigade name broken across lines lost its tail in the masked text.

Masks: naval ranks get naval masks; the feminine genitive of an
adjective surname shares the stem of its other forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01XT6iUWaQgahXDB9TWX9Bq7
Generated-With: Claude Code 2.1.42
@click0
click0 merged commit aaaf3e7 into main Nov 8, 2026
16 checks passed
@click0 click0 mentioned this pull request Nov 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants