Repository navigation
Conversation
Bumps cmarkly to the commit that lets S_out copy runs of characters the renderer writes back unchanged (LITERAL escaping, or the plaintext renderer whose outc is the identity) with a single put, instead of decoding, re-encoding and appending each code point. Spaces when wrapping, newlines in LITERAL, invalid UTF-8 and extensions with a commonmark_escape_func still take the per-character path, and a run never starts once a break opportunity is pending, so wrapping is unchanged. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bumps cmarkly to the commit that stops runs on these two code points, which cmark_utf8proc_encode_char writes as a single byte. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Hi Samuel, here is the markly half of socketry/cmarkly#7.
Node#to_plaintextspent most of its time in cmark'sS_out, which decodes each code point, re-encodes it through the renderer'soutcand appends it alone, although the plaintext renderer writes every character back unchanged. With the cmarkly change, runs of such characters are validated and copied in one append. This PR bumps thecmarklysubmodule to that commit, syncs the three files it touches intoext/markly(render.c,render.h,plaintext.c, identical to cmarkly's), adds specs and a release note.Markly.parse(text).to_plaintextgoes from 22.8 µs to 13.7 µs per text (1.66x) on 916 real texts from our app (French product descriptions and posts, median 927 bytes), and from 449k to 222k instructions per text (−51%). On 1,084 Markdown files from 265 installed gems (READMEs, changelogs, docs) it goes from 224 µs to 170 µs (1.31x, −34% instructions); onbenchmark/sample.mdfrom 146 µs to 96 µs (1.52x, −24%); on cmark'sspec.txtfrom 5.03 ms to 3.47 ms (1.45x, −35%).render_htmlis unchanged (−0.4% instructions), as expected since it doesn't go throughS_out. Instructions retired come from/usr/bin/time -l, median of 5; times are the median of 5 interleaved rounds of 9 passes; Ruby 4.0.9, Apple M5, macOS, marklymainat 436de44. The machine wasn't quiet enough for a meaningful benchmark-ips run, hence the interleaving.Outputs are byte-identical to
mainon 1,253,988 renders: 34,832 inputs (the real texts, 30,000 random token soups, 3,000 random byte strings, whitespace-mangled copies) through 4 flag and extension sets,to_plaintextandto_commonmarkat widths 0, 8, 30 and 80, andto_html, plus API-built nodes with invalid UTF-8. The C side was checked further on 8.3M outputs, detailed in the cmarkly PR. The specs pin the edge cases of the new path: wrapping inside a list item, multibyte characters, a code block's prefix in a list item, and invalid UTF-8 set throughstring_content. Like the C test, they also pass onmain, since nothing observable is supposed to change.One tradeoff to flag: until the cmarkly PR is merged, the submodule points at a commit on my fork. If it lands as a squash I'll bump it again to the merged commit. The C change itself means
S_outkeeps its column bookkeeping in two places, as discussed over there.The same change is also proposed to GitHub's cmark-gfm in github/cmark-gfm#397.
Two notes from the review of the cmarkly PR, both already in this branch: U+FFFE and U+FFFF stop a run, because cmark encodes them as a single byte (the first version copied their three bytes, which my fuzzing missed), and the gain is for the default width of 0; with
to_plaintext(width: 80)it drops to about 10%, since a run stops at every break opportunity.🤖 Generated with Claude Code, but prompted, read, and verified by @czj, a caring human :)