Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
43 changes: 43 additions & 0 deletions .github/workflows/codspeed.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,43 @@
name: codspeed

on:
push:
branches:
- main
pull_request:
branches:
- main
# Lets CodSpeed trigger backtest runs to build the initial history.
workflow_dispatch:

permissions:
contents: read
id-token: write # OIDC authentication with CodSpeed, no token secret needed

concurrency:
group: ${{ github.workflow }}-${{ github.event.pull_request.number || github.sha }}
cancel-in-progress: true

jobs:
benchmarks:
name: Run benchmarks
runs-on: ubuntu-latest
timeout-minutes: 30

steps:
- uses: actions/checkout@v7
Comment thread
coderabbitai[bot] marked this conversation as resolved.
with:
persist-credentials: false
- uses: actions/setup-node@v7
with:
package-manager-cache: false
node-version: '24.21.0'
- run: npm i -g --force corepack && corepack enable
# `postinstall` stubs every package `dist/` to its sources, so the
# benches measure the checked-out code without a build step.
- run: pnpm install
- name: Run benchmarks
uses: CodSpeedHQ/action@v5
with:
mode: simulation
run: pnpm bench
13 changes: 12 additions & 1 deletion AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -37,7 +37,7 @@ This is a **monorepo** containing the Comark Markdown parser, document model, pl
│ └── 4.ai/ # AI streaming examples (Nuxt + AI SDK)
├── docs/ # Documentation site (comark-docs layer; llms topic files in docs/server/routes/llms-*.txt.get.ts)
├── playground/ # Nuxt playground (`pnpm dev:playground`)
├── benchmarks/ # mitata benchmarks for parse/render/plugins
├── benchmarks/ # CodSpeed Vitest benches (*.bench.ts) + mitata cross-library comparisons (*.ts)
├── scripts/ # Build/sync/release scripts
├── test/ # Root-level tests (bundle-size snapshot)
├── pnpm-workspace.yaml # Workspace configuration + dependency catalog
Expand Down Expand Up @@ -896,6 +896,7 @@ pnpm build # Build all packages, then sync plugin re-exports
pnpm stub # Point every package's dist/ at src/ for local dev (runs on postinstall)
pnpm test # Run all package tests
pnpm test:spec # Run the SPEC fixtures only
pnpm bench # Run the CodSpeed Vitest benches (benchmarks/*.bench.ts; wall-clock locally)
pnpm lint # oxlint + oxfmt --check
pnpm lint:fix # oxlint --fix + oxfmt (write)
pnpm typecheck # tsc --noEmit against the root tsconfig
Expand Down Expand Up @@ -923,10 +924,20 @@ Workflows live in `.github/workflows/`:
| Workflow | Purpose |
|----------|---------|
| `ci.yml` | lint → prepack → test → publish preview (pkg.pr.new) → bundle size check |
| `codspeed.yml` | Runs `pnpm bench` under CodSpeed CPU simulation; reports perf changes on PRs |
| `commit-signature.yml` | Fails PRs containing unsigned commits |
| `bundle-snapshot.yml` | Reports bundle-size snapshot drift and updates it on demand |
| `docs-preview-comment.yml` | Posts a docs preview link on PRs that touch `docs/content/**` |

### Benchmarks (CodSpeed)

`benchmarks/*.bench.ts` run through `benchmarks/vitest.config.ts` with
`@codspeed/vitest-plugin` (a no-op outside CodSpeed). Shared inputs live in
`benchmarks/fixtures.ts`. Renaming a bench or editing a fixture resets its
history, so add new ones instead. Warm parsers at module load. Streaming
benches need a dedicated parser and a `{ streaming: false }` reset inside the
bench body; otherwise the incremental cache returns the previous result.

### Bundle size snapshot

`test/bundle.test.ts` asserts an inline snapshot of the published size of every
Expand Down
19 changes: 19 additions & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -91,6 +91,25 @@ pnpm vitest run bundle

If your change legitimately alters package sizes, update the snapshot with `pnpm vitest run bundle -u` after building, and review the resulting diff. Restore source stubs with `pnpm stub` before returning to example development.

## Benchmarks

Parse, streaming, render and plugin performance is tracked with [CodSpeed](https://app.codspeed.io/comarkdown/comark). The `codspeed` workflow runs the Vitest benches in `benchmarks/*.bench.ts` on every pull request and comments with any change compared to `main`. CodSpeed measures in CPU simulation, so results do not depend on CI machine noise.

Run the suite locally (wall-clock timings, which are only useful for rough comparisons on your machine):

```sh
pnpm bench
pnpm bench parse # only benchmarks/parse.bench.ts
```

When adding a benchmark:

- Put shared inputs in `benchmarks/fixtures.ts`. Renaming a benchmark or editing a fixture resets its history in CodSpeed, so prefer adding a new one.
- Create parsers once at module level and call each one before the benches run, so samples measure steady-state work, not setup.
- Give every streaming bench its own parser and reset it with a non-streaming call inside the bench. Otherwise the incremental cache returns the previous result without parsing.

The `mitata` scripts in `benchmarks/*.ts` compare Comark with markdown-it, markdown-exit and remark. Run them directly, for example `pnpm tsx benchmarks/comark-parse.ts`; they are not part of CodSpeed.

## Prepare a pull request

Link the relevant issue and follow the pull request template's **What** and **Why** sections. Describe the user-visible problem, the chosen fix and any compatibility impact. Include the checks you ran and any remaining limitations.
Expand Down
Loading
Loading