Repository navigation
Speed up documents with many lists - #772
Open
deadstrobe5 wants to merge 2 commits into
Open
deadstrobe5 wants to merge 2 commits into
deadstrobe5 wants to merge 2 commits into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
_do_lists()takes quadratic time on documents with many lists. After every list it converts, it searches the rest of the text once for the next unordered list and once for the next ordered list, and it splices the converted list back into the full text. A document that only uses one list style rescans to the end for the other style every time, and every splice copies the whole document.This change finds the next list marker with one small pattern, tries the unordered and ordered patterns at that position, and collects the converted lists in a list of pieces that is joined once at the end. A converted list always ends with a newline (a blank line at the top level), so right after a list the pattern is tried without the blank-line prefix. That keeps an unordered list directly followed by an ordered one (#16) working as before. The output is unchanged.
RELEASES.md, first 200 KBapi.md(153 KB)README.md(83 KB)Median of 5 alternating rounds in one process (21 for the small input) with the old module loaded next to the new one, CPython 3.14, Linux.
python test.py -- -knownfailurepasses, and the output matches master on 86,000 random inputs (lists, indentation, code spans, inline HTML, links, backslash escapes,safe_modeand common extras). The new ReDoS casemany_liststakes 0.6 s here and 7.2 s on master. The newtm-cases/many_listscase covers lists separated by paragraphs, a horizontal rule, a numbered line inside a paragraph, and an unordered list directly followed by an ordered one. Its expected HTML is what master produces.