Repository navigation
Refresh the large-bibliography benchmark results (Phase A4) - #153
Merged
Merged
Conversation
Re-run the PHP formatter (scripts/benchmark-formatter.php) and JS orchestration (npm run benchmark:perf) benchmarks on current main, after the CSL style rewrite and particle-aware sorter. Notes the Linux container vs. May macOS difference and adds a comparison section. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01KbVuMPN7YTcV27vwsG3LZ2
dknauss
marked this pull request as ready for review
October 14, 2026 23:18
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #153 +/- ##
=========================================
Coverage 88.28% 88.28%
Complexity 22 22
=========================================
Files 61 61
Lines 7290 7290
Branches 690 690
=========================================
Hits 6436 6436
Misses 291 291
Partials 563 563 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Phase A, ticket 4 of
docs/planning/refactor-and-standards-memo.md.This reruns the May 2026 benchmark against 1.11.0 plus
main, which now has the rewritten CSL styles and the particle-aware sort. It uses the existing harnesses:scripts/benchmark-formatter.phpfor PHP andnpm run benchmark:perffor JS, 5 runs per cell. The results doc gets new tables, an honest machine note, and a comparison section.Worst p95 at 200 entries:
Every cell is at least 5× under budget.
Against May. Absolute times are about 2–3× higher in PHP and about 1.3–1.8× higher in JS. The runs came from different machines (a Linux cloud container on a 2.1 GHz Xeon, not an M-series Mac), so the doc treats the change as direction only, not as a regression.
The one real change: author-date is now the slowest style at 200 entries (about 45 ms against notes' 36 ms), where in May they matched. That could be the rewritten style or noise. Only a same-machine run of old styles against new ones would settle it.
A follow-up this doesn't cover. This PR's sibling, #152, ran the runtime-matrix timing of a 200-entry
/formatrequest over real HTTP (nginx, PHP 8.3). It measured a p50 of about 205 ms for Chicago notes, against about 71–73 ms for author-date and IEEE. In-process, the same style formats in about 34 ms. So most of the HTTP cost for notes sits outside citeproc's render: request boot, sanitization, cache miss, or CiteProc setup for that style. It's still under the 250 ms budget, but it's the closest number to it anywhere. Worth profiling separately; it's not something this doc PR should chase.Validation
output/benchmarks/.main; no new code.Checklist
🤖 Generated with Claude Code
https://claude.ai/code/session_01KbVuMPN7YTcV27vwsG3LZ2
Generated by Claude Code