Skip to content

[P2] [Windows OCR] Add a repeatable live validation matrix for the AutoHotkey importer #48

Open

Description

@Alkwer

Context

The repository has good automated coverage for border-text matching, parser behavior, import integration, and browser workflows. Those tests do not execute public/voyage-import.ahk against the live Path of Exile UI and Windows Runtime OCR.

The remaining failure modes are platform- and environment-specific:

  • display resolution and DPI scaling;
  • Windowed versus Windowed Fullscreen;
  • quick rectangular versus exact 12-point calibration;
  • chart-grid dimensions and scroll position;
  • OCR language selection and noisy tooltip text;
  • normal versus elevated game/helper privilege levels;
  • tooltip timing, reroll-cost capture, cancellation, and temporary-file cleanup.

Goal

Define and execute a repeatable, privacy-safe live validation procedure for the AutoHotkey importer, with sanitized text fixtures for reproducible failures where practical.

Acceptance criteria

Test matrix

  • Define a minimum supported Windows 10/11 validation matrix.
  • Cover representative 1080p, 1440p, and 4K configurations and common DPI scaling values.
  • Cover Windowed and Windowed Fullscreen modes.
  • Record the OCR language configuration used for each run.
  • Verify behavior when Path of Exile and the helper run at matching and mismatched privilege levels.

Workflows

  • Verify quick border calibration and exact 12-point calibration.
  • Verify chart-grid calibration with default and adjusted row/column counts.
  • Verify F9, Ctrl+F9, F10, calibration preview, and reroll-cost synchronization.
  • Verify that incomplete/missed OCR cannot leave stale borders presented as a successful complete scan.
  • Verify screenshots, generated bridge files, and OCR output are removed after success, failure, abort, and exit.

Evidence and regression handling

  • Record results and known limitations in docs/windows-ocr.md or a dedicated validation document.
  • Convert reproducible OCR failures into minimized, sanitized text fixtures where possible.
  • Do not commit screenshots, account/character data, real user exports, or temporary OCR artifacts.
  • Keep manual border entry fully usable when the helper is unsupported or fails.
  • Run npm run validate and the relevant OCR/import Playwright tests after fixture or parser changes.

Non-goals

  • Do not automate launching or controlling the live game in GitHub Actions.
  • Do not require Windows-only validation in the normal Linux CI quality job.
  • Do not upload OCR screenshots or tooltip captures to a collection backend.

Activity

  1. Alkwer commented on Aug 2, 2026

    @Alkwer
    OwnerAuthor

    Hardening code has been merged to main in #71 (fd7ac44).

    Implemented:

    • AutoHotkey sweeps now include explicit expected/captured scan metadata.
    • New importer sweeps are transactional: interrupted or structurally incomplete scans keep the existing borders instead of producing a mixed stale snapshot.
    • A structurally complete scan clears individual unmatched positions, so an old modifier cannot be shown as part of the current roll.
    • A systemic 0/12 OCR failure preserves manually entered and previously valid borders.
    • The UI reports the expected denominator (11/12, not 11/11) and the actual Windows OCR recognizer language.
    • AutoHotkey no longer claims that 12 borders were refreshed before the browser has matched them.
    • Screenshots, generated PowerShell, and OCR output use a run-owned prefix and are cleaned after success, failure, timeout, F10, and exit.
    • Legacy single-border text fixtures remain supported.

    Validation completed:

    • npm run validate — 208 Vitest tests, 8 data tests, typecheck, lint, formatting, and production build passed.
    • Playwright — 11/11 Chromium tests passed, including the new interrupted-sweep regression.
    • GitHub Actions quality passed on Harden the Windows OCR importer #71.
    • AutoHotkey v2 smoke test exercised the generated PowerShell bridge and handled a controlled missing-image failure without leaving new run-owned temp artifacts.
    • Rendered localhost QA confirmed that an existing Divine border survives an 11/12 Chaos sweep and that the failure/language status is visible.

    This issue should remain open. The live Windows 10/11 + Path of Exile validation matrix, privilege-level combinations, display/DPI combinations, and real success/failure/abort cleanup observations still need to be executed and recorded before the remaining acceptance criteria can be checked.

  2. Alkwer commented on Aug 3, 2026

    @Alkwer
    OwnerAuthor

    Audit priority

    P2 — the Windows OCR importer lacks a repeatable live validation matrix, leaving an important ingestion path insufficiently verified.

    Priority assigned during the fork audit of revision f78a0ecb6707c8aa54263b2e9e3b403681088820.

  3. changed the title [-][Windows OCR] Add a repeatable live validation matrix for the AutoHotkey importer[/-] [+][P2] [Windows OCR] Add a repeatable live validation matrix for the AutoHotkey importer[/+] on Aug 7, 2026
  4. Alkwer commented on Aug 7, 2026

    @Alkwer
    OwnerAuthor

    Live validation update — 2026-08-07

    This issue should remain open. The repeatable matrix/preflight work is prepared locally, and the following observations were executed without committing screenshots, OCR text, clipboard contents, account data, character data, or temporary artifacts.

    Executed

    • Preflight passed on Windows 11 25H2 build 26200.8894, AutoHotkey 2.0.19, 3840×2160, 150% DPI, Windowed Fullscreen, with both the game and helper running non-elevated.
    • Installed Windows OCR languages were en-US and pl.
    • Quick border, default chart-grid/tab, and reroll coordinates were detected; exact 12-point calibration was not present.
    • Ctrl+F4 traversed the saved quick-calibration points without creating OCR artifacts.
    • A board-only inspection confirmed that the visible board had no active Corruption Currents, so there were no tooltip texts available for a successful scan.
    • Ctrl+F9 exercised the live capture/OCR failure path: no clipboard payload was produced, the solver remained at 0/12 borders, and the run-owned %TEMP% artifact count returned to zero.
    • F10 interrupted a live Ctrl+F9 capture: no partial payload was pasted and the run-owned artifact count returned to zero.
    • Manual fallback remained usable after failure: a border was entered directly (1/12) and cleared again (0/12).
    • npm run validate passed 45 Vitest files / 345 tests, the data suite, typecheck, lint, line endings, formatting, and the production build.
    • The interrupted Windows OCR Playwright regression passed in Chromium.
    • The user's original downloaded helper and its unchanged calibration were restored after validation.

    Still blocked / not recorded as passes

    • successful quick and exact 12/12 OCR scans require an active Voyage with all 12 Corruption Currents visible;
    • F9 with default and adjusted chart grids;
    • reroll-cost synchronization on a successful live scan;
    • stale-snapshot protection following a successful baseline scan;
    • Windows 10, 1080p, 1440p, Windowed mode, and the full matching/mismatched privilege matrix.

    No unexecuted cell is being treated as a pass. The next live session needs an existing active Voyage plus the remaining Windows/display/privilege environments before the acceptance checklist can be completed.

  5. Alkwer commented on Aug 10, 2026

    @Alkwer
    OwnerAuthor

    PR #233 adds the dedicated, privacy-safe live validation matrix and run sheet. It covers Windows 10/11, 1080p/1440p/4K and DPI, Windowed/Windowed Fullscreen, OCR language, matched/mismatched privileges, quick/exact calibration, default/adjusted grids, F9/Ctrl+F9/F10/preview/reroll, stale-snapshot behavior, manual fallback, and cleanup after success/failure/abort/exit. The earlier 2026-08-07 partial Windows 11/4K observations remain valid, but no new live row was executed here: this environment currently has neither a running Path of Exile process nor AutoHotkey. Keep #48 open until all eight minimum rows have reviewed Pass run sheets; PR #233 intentionally does not auto-close it.

  6. reopened this on Aug 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions