A benchmark suite comparing execution time and source size across programming languages, with an interactive explorer for results and source code.
The catalog contains 63 execution variants across 54 language families. The suite covers ten workloads: binary trees, fannkuch-redux, FASTA, k-nucleotide, Mandelbrot, N-body, pi digits, regex-redux, reverse complement, and spectral norm. Implementations span interpreted, JIT-compiled, and ahead-of-time compiled execution. Results include individual timing samples, source hashes, output validation, and measurement provenance.
Compare runtime and lines of code, inspect programs and their recorded samples, and filter by workload, language, release year, or language characteristics. Chart layers show individual samples, language summaries, density, and global statistics. Ratios use each workload's fixed best recorded baseline.
The Data view lists every recorded run across the full retained implementation inventory. Sort columns, search, apply the sidebar filters, or select individual rows. Implementation links open source and timing details. Download the entire dataset, filtered rows, or selected rows as CSV or JSON; JSON includes matching source files and program metadata. Exports cover all pages and retain full numeric precision. Shared timing aliases identify the same recorded observations once. Copy link preserves the filters, sort, page, row selection and open source.
The methodology describes timing boundaries, source counting, library use, and comparison rules. Mech's optimization history connects compiler and runtime changes to their measured effects.
The implementation audit separates unchanged published submissions from local implementations and adaptations, records source reviews and measured improvements, and identifies remaining optimization questions. Fastest and smallest endpoints are winners among the retained qualified candidates; further improvements remain possible.
Complementary source measures expose application and submitted bytes, deterministic gzip size and pinned BPE token counts alongside retained application LOC, qualified formatted LOC and qualified logical units. The six chart options keep unavailable counts blank. Native logical-source qualification covers 74 exact programs under the separate Python/Julia and Clojure owner policies; other source families await equivalent checks. Each measure has explicit limits.
Reproduction records expose recorded versions, build commands, runtime settings, inputs and gaps. Implementation authorship distinguishes published sources, local AI adaptations and generated programs. The data view and exports include verified model IDs, original attribution, license terms and explicit evidence gaps.
| Directory | Contents |
|---|---|
| benchmarks/ | Workload definitions and chart baselines |
| programs/ | Measured source bundles organized by language and workload |
| results/current/ | Program results and individual timing samples |
| data/ | Explorer dataset and language metadata |
| site/ | Interactive explorer source and tests |
| docs/ | Methodology, language classifications, and optimization studies |
| experiments/ | Qualification protocols, active studies, and publication evidence organized by implementation |
| archive/ | Detailed measurement reports and experimental evidence |
Python 3 is sufficient to build and serve the explorer:
python3 tools/serve.py --port 8766Open http://127.0.0.1:8766/benchmarks/. To generate the static site without starting a server:
python3 tools/build_site.pyGenerated files are written to site/public/.
python3 tools/export_catalog.py
python3 tools/test_export_catalog.py
python3 tools/verify_repository.py
python3 tools/test_build_site.py
python3 tools/test_authorship.py
python3 tools/audit_authorship.py --check
python3 tools/build_site.py
node site/tests/data_projection.mjs data/benchmarks.json site/public/data.json
node site/tests/layers.mjs
node site/tests/rankings.mjs
node site/tests/languages.mjs data/benchmarks.json
node site/tests/publication.mjs
node site/tests/benchmark_traits.mjs
node site/tests/density.mjs
node site/tests/view_state.mjs
node site/tests/sample_data.mjsWith Playwright and Chrome installed, node site/tests/sample-table.mjs site/public
checks the table, source links, desktop/mobile layouts, shared selections and
downloaded CSV/JSON contents in isolated browser contexts.
These commands export and check recorded results. Running the benchmarks requires the toolchains and dependencies specified in the measurement reports. Measurements run sequentially, without competing benchmark or compilation jobs.
The explorer performance audit describes browser payload projection, rendering changes and their validation. Complete measurement evidence remains in the canonical catalog.
See coverage status for published results and remaining qualification work.
Repository code is licensed under Apache-2.0. Third-party benchmark submissions retain their respective licenses and attribution.