Skip to content

Copy verbatim runs in the plaintext renderer, 2x faster parse + plaintext - #7

Open
czj wants to merge 2 commits into
socketry:mainfrom
czj:perf/plaintext-verbatim-runs
Open

czj wants to merge 2 commits into
socketry:mainfrom
czj:perf/plaintext-verbatim-runs

Conversation

@czj

@czj czj commented Oct 26, 2026 •

Copy link
Copy Markdown

Hi Samuel, thanks for keeping cmark-gfm alive through cmarkly and markly, we render all our product texts with it.

While profiling to_plaintext I noticed that S_out in render.c decodes every code point, calls outc, which re-encodes it, and appends it to the buffer one at a time. For the plaintext renderer outc is the identity, and every renderer writes LITERAL text back unchanged too, so most of that work rebuilds the bytes it started from. This PR lets S_out validate a run of such characters and copy it with a single cmark_strbuf_put, updating column, begin_line and begin_content the way the per-character loop would. Spaces when wrapping, newlines in LITERAL, invalid UTF-8 and extensions with a commonmark_escape_func still take the existing path, and a run never starts once a break opportunity is pending, so wrapping is unchanged. plaintext.c passes a NULL outc to cmark_render to opt in; render.h is internal and not installed, so the public API and ABI don't move.

Parsing then rendering to plain text with CMARK_OPT_DEFAULT goes from 36.9 µs to 18.7 µs per text (1.97x) on 916 real texts from our app (French product descriptions and posts, median 927 bytes), and from 385k to 163k instructions per text (−58%). On 1,084 Markdown files taken from 265 installed Ruby gems (READMEs, changelogs, docs) it goes from 1.87M to 1.10M instructions per text (−41%). Instructions retired come from /usr/bin/time -l, median of 5; times are the median of 5 interleaved rounds; Apple M5, macOS, Apple clang 21.0.0, CMAKE_BUILD_TYPE=Release, main at 85ef5cf.

The output is byte-identical to main: 66,832 inputs (the 916 real texts, 60,000 random soups of Markdown tokens, 5,000 random byte strings, whitespace-mangled copies of the real texts) through 5 option and extension sets, rendered to plaintext, commonmark, latex and man at widths 0, 1, 5, 8, 30 and 80 plus HTML, along with API-built text, code and code block nodes holding invalid UTF-8: 8,354,225 outputs with the same SHA-256 on both sides. A mutant that miscounts the column of long runs changes that hash, so the harness does exercise the new path. The new api_test case covers wrapping inside a list item, multibyte characters and a code block's prefix; I should be upfront that it passes on main as well, since the change is meant to be invisible.

The tradeoff is that S_out now keeps its bookkeeping in two places: anyone changing the per-character loop later has to mirror it in the run path. I kept the run path to about 30 lines next to the loop for that reason.

The markly side (submodule bump, synced ext/markly, specs) is in socketry/markly#35, and the same commit is proposed to GitHub's cmark-gfm in github#397, in case you'd rather it reach cmarkly through an upstream sync. Would you like this listed under the readme's maintained differences while the upstream one is open?


🤖 Generated with Claude Code, but prompted, read, and verified by @czj, a caring human :)

S_out decoded every code point, re-encoded it and appended it to the
buffer one at a time, even when the renderer writes it back unchanged
(LITERAL escaping, or the plaintext renderer whose outc is the identity).
Runs of such characters are now validated and copied with a single put.
Spaces when wrapping, newlines in LITERAL, invalid UTF-8 and extensions
with a commonmark_escape_func still take the per-character path, and a
run never starts once a break opportunity is pending, so wrapping is
unchanged.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
cmark_utf8proc_encode_char writes these two code points as a single byte, so copying their original three bytes changed the output.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@czj

czj commented Oct 27, 2026

Copy link
Copy Markdown
Author

A correction to the description: the output was not byte-identical for U+FFFE and U+FFFF. cmark_utf8proc_encode_char writes those two code points as a single byte, while a run copied their original three bytes, and my fuzzing never generated them. 23f5ae2 stops a run on either one; with it, outputs match main again on the same 8.35M renders, plus 61M random lines that now include both code points.

One more thing I should have said upfront: the 2x is at width 0, which is markly's default for to_plaintext. With wrapping at 80 columns the gain drops to about 10%, because a run stops at every break opportunity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant