Fixed contextName stack exception for context-less PDFs when using PdfTextLocator

[ConvoluteHumanBot]

Bugfix

Fixed emptystack exception in PdfContentTextLocator.java, contextNames stack was not tracked on pushContext function call, causing context-less documents to throw exception.
Changed internal mode matching strategy from int to enum for better readability.

Your real name

Alessandro Ragusi (ConvoluteHumanBot)

[codacy-production]

Up to standards ✅

🟢 Issues 0 issues

Results:
0 new issues

View in Codacy

🟢 Metrics 47 complexity · -143 duplication

Metric	Results
Complexity	47
Duplication	-143

View in Codacy

_{NEW Get contextual insights on your PRs based on Codacy's metrics, along with PR and Jira context, without leaving GitHub. Enable AI reviewer
TIP This summary will be updated as you push new changes.}

[sonarqubecloud]

Quality Gate failed

Failed conditions
27.9% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

[andreasrosdal]

This PR might actually be a good idea, I think it could be interesting to improve this and get it included. Do you have more information? @ConvoluteHumanBot

[ConvoluteHumanBot]

Hi Andreas,
This PR was a fix for some PDF that would have caused an ugly EmptyStackException (will add those to the tests) when pushing and popping contextNames.
I'm currently working on a cleaner PR that includes more fixes, namely:

• The actual width calculation for DocumentFont was still broken, because the widths array was not accessed properly.
• Concatenating multiple PdfStrings in the same line to perform a more precise pattern matching. The idea is similar to the algorithm in the renderText function in MarkedUpTextAssembler, this does not break previous cases but adds more precision for complex documents.

About the DocumentFont problem: CMapAwareDocumentFont is properly parsing the CID map to decode the chars, but the widths are stored sequentially as they appear in the PdfName.TOUNICODE stream. The char width is then retrieved using the unicode integer, and was causing a mismatch. I'm trying to figure out if it is better to add a map from UNICODE to CID and leave the widths initialization as is (currently done locally and working), or if it is better reworking the widths association altogether and use an IntHashmap as for CJK fonts.

Thanks for your time @andreasrosdal

[ConvoluteHumanBot]

Text locator: matched-substring coordinates, and a fix to glyph width lookup

Follow-up to the coordinate-based text extraction that landed earlier
(PdfTextLocator / MatchedPattern). That work could tell you which line
matched a pattern; this PR makes it report where the match itself sits, and
fixes the width lookup that the coordinates depend on.

The width fix

DocumentFont.getWidth(int) is keyed by Unicode character — its Javadoc says
"the unicode char to get the width of", and for Type0 fonts it reads the
metrics map, which fillMetrics builds keyed by Unicode. But a content stream
yields character codes, so the parser was measuring codes against a
Unicode-keyed table.

For a WinAnsi font this goes unnoticed, because code == Unicode across ASCII.
For an Identity-H font the two key spaces are unrelated, and the lookup either
misses (width 0) or — worse — hits an unrelated entry and returns a plausible
wrong number. Measured on a generated Identity-H document:

U+0069 cid=76 realWidth=222 | getWidth(cid=76)=556   <- U+004C's width, silently wrong
U+0061 cid=68 realWidth=556 | getWidth(cid=68)=0

Bridging this with a reverse Unicode→CID map is lossy: several codes can share
one Unicode character, one code can decode to several characters (a ligature),
and a code need not decode to anything at all.

The fix goes the other way, which is how a PDF actually stores widths.
DocumentFont.readWidths() already parses /W into a CID-keyed IntHashtable
— and then discards it after inverting it into metrics. This PR keeps it,
along with /DW, and adds a code-keyed accessor beside the Unicode-keyed one:

public int getWidthOfCode(int code)

getWidth(int) is untouched, so no existing caller changes behaviour. The
parser and the locator now measure through getWidthOfCode, which also means
widths work for a Type0 font with no /ToUnicode map, where metrics is
never populated and every width is currently 0.

Before / after

Locating patterns in an Identity-H document, via MatchedPattern#getMatchedBBox:

pattern	before	after
H	x1=53.99, width=0.00	x1=80.52, width=8.66
Hello	x1=53.99, width=8.66	x1=80.52, width=27.34
Hello Wonderful	x1=53.99, width=17.33	x1=80.52, width=85.32

Before, only spaces had a width, via a fallback; every real glyph measured 0, so
the offsets collapsed and the left edge was wrong. After, all three matches
share a left edge and each is wider than the shorter match it contains. Checked
against the /W array, 17 of 17 glyph widths now resolve correctly by code.

Also fixed in the parser

• Text matrix advance. When displayPdfString was pulled up into
  PdfContentStreamHandler.parsePdfString, the advance changed from
  getUnscaledTextWidth(graphicsState()) to getWidth(). Those are different
  quantities — getWidth() is user space (post-CTM), while advancing the text
  matrix needs text space. Under a scaling CTM the advance came out short and
  the assembler inserted spurious spaces (TEST extracted as T E ST).
• Two-byte code points. A two-byte encoding packs each code into two chars
  of the PdfString, so it cannot be walked a char at a time. Measuring and
  word-splitting go back through PdfString#getOriginalChars(), which does the
  pairing. ParsedText.create previously measured the raw string, giving every
  Identity-H run twice as many (wrong) glyphs.
• Offset alignment in the locator. The locator runs a Matcher over decoded
  text and maps start()/end() back through an offset list, so offsets must
  stay index-aligned with the decoded string. Decoded text and offsets are now
  built in one pass over the codes, which makes the invariant structural and
  handles the two cases the old loop could not: a code that decodes to nothing
  (the pen advances, no text is added) and a code that decodes to several
  characters (its advance is spread across them).
• Consolidated three copies of the advance formula into
  ParsedText.advanceForCode, so word offsets and the text matrix advance can
  no longer drift apart. Removed an unused ParsedText constructor that held a
  fourth copy.

New public API

• DocumentFont#getWidthOfCode(int) — width of a character code, in 1000ths of
  a text space unit.
• MatchedPattern#getMatchedText() / #getMatchedBBox() — the matched
  substring and its box, as opposed to the containing line and #getCoordinates().
• ParsedText#getCodePoints() — the character codes of a fragment.

Testing

./mvnw test is green across all modules (2081 tests in openpdf-core), and
the checkstyle goal reports no violations.

Added PdfTextExtractorTest#testTextLocatorCoordinatesAreProportionalWithIdentityHFont,
which asserts that matches beginning at the same character share a left edge and
that a longer match is wider than the shorter one it contains. It fails without
this change with A single character should have a positive width, got: 0.0.

Known limitations

• Fonts using a predefined CMap (/Encoding /GBK-EUC-H and friends) go
  through cjkMirror, which is Unicode-keyed. Mapping code → CID there needs
  the encoding CMap rather than the ToUnicode one, so getWidthOfCode falls
  through to the existing behaviour for those. No regression — that case is
  equally affected today — but it is not fixed here.
• Splitting a ligature's advance evenly across the characters it decodes to is
  an approximation. It is bounded within a single glyph, and there is no finer
  position available.

[sonarqubecloud]

Quality Gate passed

Issues
0 New issues
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

[ConvoluteHumanBot]

@andreasrosdal Hi, the code is now clean and the width conversion for glyphs is tested using the proper CID map. Claude spotted another possible failure on a restricted set of fonts, i should address it on a next PR.