Skip to content

feat: harden Auto Redact and export for GDPR, fewer false positives - #131

Merged
overwrite00 merged 8 commits into
developfrom
feat/autoredact-gdpr-hardening
Nov 10, 2026
Merged

overwrite00 merged 8 commits into
developfrom
feat/autoredact-gdpr-hardening

Conversation

@overwrite00

Copy link
Copy Markdown
Owner

Summary

Reduces false positives in the automatic scan (ordinary Italian words, fragments of neighbouring lines) and hardens the export and the pseudonymization workflow.

  • Evidence-based scoring of names/places with an Italian lexicon and ISTAT place names; uncertain hits become reviewable suggestions (dialog and page preview).
  • Health-document profile: birth dates, health card, prescription number, exemption codes, postcodes, house numbers, signature/stamp suggestions.
  • Redaction rectangles no longer damage neighbouring lines; placeholders are drawn after the redactions (compact label in small boxes, restore map v4).
  • Export sanitiser and output verification, no unredacted temp copy, neutral file names, audit record, local data wipe.
  • Encrypted dictionaries with expiry, sealed pseudonym key, restore map envelope v3 (Argon2id, optional recovery key), pseudonym styles.
  • Prerelease label bumped to beta.2.

Compatibility notes

  • Restore maps created by this version cannot be opened by 2.4.0-beta.1 or earlier; older maps still open.
  • On first start the plaintext blocklist.txt / allowlist.txt are migrated to encrypted files and overwritten.

Testing

  • pytest tests/: 309 passed
  • mypy on the 12 modules listed in the workflows: clean
  • Offscreen GUI tests with real mouse events for the review, overlay, export and restore flows

Detection
- Evidence-based scoring of PERSON/LOCATION hits with an Italian lexicon,
  accepted/suggested states and a review dialog, preview overlay and
  health-document profile (birth dates, health card, prescription number,
  exemption codes, postcodes, house numbers, signature/stamp suggestions).
- Validation of weak identifier patterns, allowlist by word identity,
  ligature/letter-spaced/hyphenated word handling, ISTAT place names.

Redaction geometry
- Rectangles stop short of neighbouring lines' glyph cores (measured
  deletion rule, pinned by a test); per-character coverage check; text under
  a box is read from a single baseline.
- Placeholders drawn after the redactions, with compact labels in small
  boxes (restore map v4).

Export and privacy
- Export sanitiser and output verification, no unredacted temp copy,
  neutral file names, audit record, local data wipe, rotating logs.
- Encrypted dictionaries with expiry, sealed pseudonym key, restore map
  envelope v3 (Argon2id, optional recovery key), pseudonym styles.

Also bumps the prerelease label to beta.2 and aligns the mypy module list
across workflows.
@overwrite00 overwrite00 added type:feature New feature or request lang:python Python-related scope:core Core library/framework changes labels Nov 8, 2026
@overwrite00 overwrite00 self-assigned this Nov 8, 2026
@github-actions github-actions Bot added type:docs Improvements or additions to documentation type:ci CI/CD, GitHub Actions labels Nov 8, 2026
@overwrite00
overwrite00 merged commit 0fa62ff into develop Nov 10, 2026
4 checks passed
@overwrite00
overwrite00 deleted the feat/autoredact-gdpr-hardening branch November 10, 2026 17:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

lang:python Python-related scope:core Core library/framework changes type:ci CI/CD, GitHub Actions type:docs Improvements or additions to documentation type:feature New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant