Skip to content

Speed up to_plaintext 1.3–1.7x by copying runs of unchanged text - #35

Open
czj wants to merge 2 commits into
socketry:mainfrom
czj:perf/plaintext-verbatim-runs
Open

czj wants to merge 2 commits into
socketry:mainfrom
czj:perf/plaintext-verbatim-runs

Conversation

@czj

@czj czj commented Oct 27, 2026

Copy link
Copy Markdown

Hi Samuel, here is the markly half of socketry/cmarkly#7.

Node#to_plaintext spent most of its time in cmark's S_out, which decodes each code point, re-encodes it through the renderer's outc and appends it alone, although the plaintext renderer writes every character back unchanged. With the cmarkly change, runs of such characters are validated and copied in one append. This PR bumps the cmarkly submodule to that commit, syncs the three files it touches into ext/markly (render.c, render.h, plaintext.c, identical to cmarkly's), adds specs and a release note.

Markly.parse(text).to_plaintext goes from 22.8 µs to 13.7 µs per text (1.66x) on 916 real texts from our app (French product descriptions and posts, median 927 bytes), and from 449k to 222k instructions per text (−51%). On 1,084 Markdown files from 265 installed gems (READMEs, changelogs, docs) it goes from 224 µs to 170 µs (1.31x, −34% instructions); on benchmark/sample.md from 146 µs to 96 µs (1.52x, −24%); on cmark's spec.txt from 5.03 ms to 3.47 ms (1.45x, −35%). render_html is unchanged (−0.4% instructions), as expected since it doesn't go through S_out. Instructions retired come from /usr/bin/time -l, median of 5; times are the median of 5 interleaved rounds of 9 passes; Ruby 4.0.9, Apple M5, macOS, markly main at 436de44. The machine wasn't quiet enough for a meaningful benchmark-ips run, hence the interleaving.

Outputs are byte-identical to main on 1,253,988 renders: 34,832 inputs (the real texts, 30,000 random token soups, 3,000 random byte strings, whitespace-mangled copies) through 4 flag and extension sets, to_plaintext and to_commonmark at widths 0, 8, 30 and 80, and to_html, plus API-built nodes with invalid UTF-8. The C side was checked further on 8.3M outputs, detailed in the cmarkly PR. The specs pin the edge cases of the new path: wrapping inside a list item, multibyte characters, a code block's prefix in a list item, and invalid UTF-8 set through string_content. Like the C test, they also pass on main, since nothing observable is supposed to change.

One tradeoff to flag: until the cmarkly PR is merged, the submodule points at a commit on my fork. If it lands as a squash I'll bump it again to the merged commit. The C change itself means S_out keeps its column bookkeeping in two places, as discussed over there.

The same change is also proposed to GitHub's cmark-gfm in github/cmark-gfm#397.

Two notes from the review of the cmarkly PR, both already in this branch: U+FFFE and U+FFFF stop a run, because cmark encodes them as a single byte (the first version copied their three bytes, which my fuzzing missed), and the gain is for the default width of 0; with to_plaintext(width: 80) it drops to about 10%, since a run stops at every break opportunity.


🤖 Generated with Claude Code, but prompted, read, and verified by @czj, a caring human :)

czj and others added 2 commits October 27, 2026 07:21
Bumps cmarkly to the commit that lets S_out copy runs of characters the
renderer writes back unchanged (LITERAL escaping, or the plaintext
renderer whose outc is the identity) with a single put, instead of
decoding, re-encoding and appending each code point. Spaces when
wrapping, newlines in LITERAL, invalid UTF-8 and extensions with a
commonmark_escape_func still take the per-character path, and a run
never starts once a break opportunity is pending, so wrapping is
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bumps cmarkly to the commit that stops runs on these two code points, which cmark_utf8proc_encode_char writes as a single byte.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant