Skip to content

Speed up documents with many indented code blocks - #775

Open
deadstrobe5 wants to merge 2 commits into
trentm:masterfrom
deadstrobe5:perf/code-block-lookahead
Open

deadstrobe5 wants to merge 2 commits into
trentm:masterfrom
deadstrobe5:perf/code-block-lookahead

Conversation

@deadstrobe5

Copy link
Copy Markdown

The _do_code_blocks() regex ends with a negative lookahead, (?!([^<]|<(/?)span)*\</code\>), that keeps it from matching inside an existing <code> element (for syntax highlighting). After every indented code block, that lookahead scans forward to the next < that isn't a <span>. Plain Markdown has no such tag before the end of the document, so each block scans the rest of the text, and a document with many indented code blocks takes quadratic time.

The lookahead can only match if the text contains </code>, so this change adds it to the pattern only in that case. Without </code> in the text, the lookahead could never match, so dropping it gives the same result.

Input Before After Speedup
500 paragraphs, each followed by an indented code block 364 ms 25 ms 14.7x
many_code_blocks, the case this PR adds to test_redos.py (3000 blocks) 7.55 s 0.13 s 60x
Markdown Syntax doc from the test suite (26 KB, has </code>) 18.4 ms 18.2 ms same pattern, within noise

Median of 5 alternating rounds in one process (3 for the 3000-block case), with the old module loaded next to the new one, CPython 3.14, Linux, on a busy machine. python test.py -- -knownfailure passes, and the output matches master on 86,000 random inputs (lists, indentation, code spans, inline HTML, links, backslash escapes, safe_mode and common extras). Documents that contain </code> keep the lookahead, so their timing does not change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant