Refactored cli - #4
BurnedChris wants to merge 22 commits into
Conversation
…consentio/benchmark' package for core benchmarking logic, replace '@cookiebench/cli' with 'cookiebench' in various benchmark configurations, and enhance package.json scripts for improved functionality. Remove deprecated CLI files and streamline project structure.
…duce interactive multi-select mode for benchmark execution, add scores command to view existing results, and integrate Perfume.js for enhanced performance metrics. Update logger utility for better CLI output and refactor commands to utilize the new logging system. Update package dependencies and documentation accordingly.
…esults to the database, and implement interactive multi-select mode for benchmark execution and results viewing. Introduce admin access checks for restricted commands, improve logging, and update documentation to reflect new features and usage instructions.
…ects. Refactor TypeScript settings, enhance package.json scripts, and improve logging utilities. Introduce new constants for better code maintainability and streamline project structure. Update documentation to reflect recent changes and ensure consistency in coding standards.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
WalkthroughNew shared, benchmark, and runner packages plus a new cookiebench CLI; legacy CLI and many in-repo collectors/aggregators removed. Benchmarks normalized (bundleType → "iife"), TypeScript configs and manifests added/updated, docs and methodology introduced, and root tooling/ignore rules adjusted. Changes
Sequence Diagram(s)%%{init: {"theme":"base","themeVariables":{"actorBorder":"#1f6feb","actorBackground":"#f0f7ff","signal":"#2d9cdb"}}}%%
sequenceDiagram
actor CLI as cookiebench-cli
participant Runner as `@consentio/runner`
participant Collectors as `@consentio/benchmark`
participant Browser as Playwright
participant Aggregator as PerformanceAggregator
CLI->>Runner: benchmarkCommand(appPath?, iterations?)
Runner->>Browser: launch browser, create context/page
Runner->>Collectors: initialize collectors (CookieBanner, Network, Resource, Perfume)
Runner->>Browser: navigate(url)
Note right of Browser: injected init scripts, Perfume, observers, route handlers
par detection & monitoring
Collectors->>Browser: setupDetection / setupMonitoring (initScript, page.route)
Browser-->>Collectors: window.__* metrics & intercepted responses
and collection
Runner->>Browser: wait for element / networkidle / traces
Runner->>Collectors: collectMetrics()
Collectors-->>Runner: CookieBannerData, NetworkMetrics, ResourceTimingData, PerfumeMetrics
end
Runner->>Aggregator: aggregateMetrics(collected data)
Aggregator-->>Runner: BenchmarkDetails
Runner->>Runner: repeat iterations, calculateAverages
Runner-->>CLI: write results.json / return BenchmarkResult
Estimated code review effort🎯 4 (Complex) | ⏱️ ~45 minutes
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches🧪 Generate unit tests (beta)
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
left a comment
There was a problem hiding this comment.
Actionable comments posted: 41
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (135)
.cursor/rules/ultracite.mdc(1 hunks).gitignore(1 hunks)benchmarks/baseline/app/layout.tsx(1 hunks)benchmarks/baseline/config.json(1 hunks)benchmarks/baseline/next-env.d.ts(1 hunks)benchmarks/baseline/next.config.ts(1 hunks)benchmarks/baseline/package.json(1 hunks)benchmarks/baseline/tsconfig.json(1 hunks)benchmarks/with-c15t-nextjs/app/layout.tsx(3 hunks)benchmarks/with-c15t-nextjs/config.json(1 hunks)benchmarks/with-c15t-nextjs/next-env.d.ts(1 hunks)benchmarks/with-c15t-nextjs/next.config.ts(1 hunks)benchmarks/with-c15t-nextjs/package.json(1 hunks)benchmarks/with-c15t-nextjs/tsconfig.json(1 hunks)benchmarks/with-c15t-react/app/layout.tsx(3 hunks)benchmarks/with-c15t-react/config.json(1 hunks)benchmarks/with-c15t-react/next.config.ts(1 hunks)benchmarks/with-c15t-react/package.json(1 hunks)benchmarks/with-c15t-react/tsconfig.json(1 hunks)benchmarks/with-cookie-control/app/layout.tsx(1 hunks)benchmarks/with-cookie-control/config.json(1 hunks)benchmarks/with-cookie-control/next.config.ts(1 hunks)benchmarks/with-cookie-control/package.json(1 hunks)benchmarks/with-cookie-control/tsconfig.json(1 hunks)benchmarks/with-cookie-yes/app/layout.tsx(1 hunks)benchmarks/with-cookie-yes/config.json(1 hunks)benchmarks/with-cookie-yes/next.config.ts(1 hunks)benchmarks/with-cookie-yes/package.json(1 hunks)benchmarks/with-cookie-yes/tsconfig.json(1 hunks)benchmarks/with-didomi/config.json(1 hunks)benchmarks/with-didomi/next.config.ts(1 hunks)benchmarks/with-didomi/package.json(1 hunks)benchmarks/with-didomi/tsconfig.json(1 hunks)benchmarks/with-enzuzo/app/layout.tsx(2 hunks)benchmarks/with-enzuzo/config.json(1 hunks)benchmarks/with-enzuzo/next.config.ts(1 hunks)benchmarks/with-enzuzo/package.json(1 hunks)benchmarks/with-enzuzo/tsconfig.json(1 hunks)benchmarks/with-iubenda/app/layout.tsx(1 hunks)benchmarks/with-iubenda/config.json(1 hunks)benchmarks/with-iubenda/next.config.ts(1 hunks)benchmarks/with-iubenda/package.json(1 hunks)benchmarks/with-iubenda/tsconfig.json(1 hunks)benchmarks/with-ketch/app/layout.tsx(1 hunks)benchmarks/with-ketch/config.json(1 hunks)benchmarks/with-ketch/next.config.ts(1 hunks)benchmarks/with-ketch/package.json(1 hunks)benchmarks/with-ketch/tsconfig.json(1 hunks)benchmarks/with-onetrust/app/layout.tsx(2 hunks)benchmarks/with-onetrust/config.json(1 hunks)benchmarks/with-onetrust/next.config.ts(1 hunks)benchmarks/with-onetrust/package.json(1 hunks)benchmarks/with-onetrust/tsconfig.json(1 hunks)benchmarks/with-osano/app/layout.tsx(1 hunks)benchmarks/with-osano/config.json(1 hunks)benchmarks/with-osano/next.config.ts(1 hunks)benchmarks/with-osano/package.json(1 hunks)benchmarks/with-osano/tsconfig.json(1 hunks)benchmarks/with-usercentrics/app/layout.tsx(1 hunks)benchmarks/with-usercentrics/config.json(1 hunks)benchmarks/with-usercentrics/next.config.ts(1 hunks)benchmarks/with-usercentrics/package.json(1 hunks)benchmarks/with-usercentrics/tsconfig.json(1 hunks)biome.jsonc(2 hunks)package.json(1 hunks)packages/benchmark/README.md(1 hunks)packages/benchmark/package.json(1 hunks)packages/benchmark/rslib.config.ts(1 hunks)packages/benchmark/src/bundle-strategy.ts(1 hunks)packages/benchmark/src/constants.ts(1 hunks)packages/benchmark/src/cookie-banner-collector.ts(1 hunks)packages/benchmark/src/index.ts(1 hunks)packages/benchmark/src/network-monitor.ts(1 hunks)packages/benchmark/src/perfume-collector.ts(1 hunks)packages/benchmark/src/resource-timing-collector.ts(1 hunks)packages/benchmark/src/types.ts(1 hunks)packages/benchmark/tsconfig.json(1 hunks)packages/cli/base.json(0 hunks)packages/cli/package.json(0 hunks)packages/cli/src/commands/benchmark/benchmark-runner.ts(0 hunks)packages/cli/src/commands/benchmark/bundle-strategy.ts(0 hunks)packages/cli/src/commands/benchmark/constants.ts(0 hunks)packages/cli/src/commands/benchmark/cookie-banner-detector.ts(0 hunks)packages/cli/src/commands/benchmark/index.ts(0 hunks)packages/cli/src/commands/benchmark/metrics-calculator.ts(0 hunks)packages/cli/src/commands/benchmark/network-monitor.ts(0 hunks)packages/cli/src/commands/benchmark/resource-collector.ts(0 hunks)packages/cli/src/commands/benchmark/types.ts(0 hunks)packages/cli/src/commands/db.ts(0 hunks)packages/cli/src/commands/results.ts(0 hunks)packages/cli/src/index.ts(0 hunks)packages/cli/src/lib/README.md(0 hunks)packages/cli/src/lib/benchmark-runner.ts(0 hunks)packages/cli/src/lib/collectors/cookie-banner-collector.ts(0 hunks)packages/cli/src/lib/collectors/index.ts(0 hunks)packages/cli/src/lib/collectors/network-monitor.ts(0 hunks)packages/cli/src/lib/collectors/resource-timing-collector.ts(0 hunks)packages/cli/src/lib/metrics/index.ts(0 hunks)packages/cli/src/lib/metrics/performance-aggregator.ts(0 hunks)packages/cli/src/lib/performance-enhanced.ts(0 hunks)packages/cli/src/lib/performance.ts(0 hunks)packages/cli/src/lib/server.ts(0 hunks)packages/cli/src/types/index.ts(0 hunks)packages/cli/src/utils/index.ts(0 hunks)packages/cli/tsconfig.json(0 hunks)packages/cookiebench-cli/README.md(1 hunks)packages/cookiebench-cli/base.json(1 hunks)packages/cookiebench-cli/package.json(1 hunks)packages/cookiebench-cli/rslib.config.ts(1 hunks)packages/cookiebench-cli/src/commands/benchmark.ts(1 hunks)packages/cookiebench-cli/src/commands/db.ts(1 hunks)packages/cookiebench-cli/src/commands/results.ts(1 hunks)packages/cookiebench-cli/src/commands/save.ts(1 hunks)packages/cookiebench-cli/src/commands/scores.ts(1 hunks)packages/cookiebench-cli/src/components/intro.ts(1 hunks)packages/cookiebench-cli/src/index.ts(1 hunks)packages/cookiebench-cli/src/types/index.ts(1 hunks)packages/cookiebench-cli/src/utils/auth.ts(1 hunks)packages/cookiebench-cli/src/utils/constants.ts(1 hunks)packages/cookiebench-cli/src/utils/index.ts(1 hunks)packages/cookiebench-cli/src/utils/logger.ts(1 hunks)packages/cookiebench-cli/src/utils/scoring.ts(27 hunks)packages/cookiebench-cli/tsconfig.json(1 hunks)packages/runner/README.md(1 hunks)packages/runner/package.json(1 hunks)packages/runner/rslib.config.ts(1 hunks)packages/runner/src/benchmark-runner.ts(1 hunks)packages/runner/src/index.ts(1 hunks)packages/runner/src/performance-aggregator.ts(1 hunks)packages/runner/src/server.ts(1 hunks)packages/runner/src/types.ts(1 hunks)packages/runner/src/utils.ts(1 hunks)packages/runner/tsconfig.json(1 hunks)tsconfig.json(2 hunks)turbo.json(1 hunks)
💤 Files with no reviewable changes (28)
- packages/cli/src/lib/metrics/index.ts
- packages/cli/src/lib/server.ts
- packages/cli/src/commands/benchmark/bundle-strategy.ts
- packages/cli/src/index.ts
- packages/cli/src/utils/index.ts
- packages/cli/src/lib/performance-enhanced.ts
- packages/cli/src/commands/benchmark/network-monitor.ts
- packages/cli/src/lib/benchmark-runner.ts
- packages/cli/src/commands/db.ts
- packages/cli/src/lib/collectors/index.ts
- packages/cli/src/lib/metrics/performance-aggregator.ts
- packages/cli/src/commands/benchmark/index.ts
- packages/cli/src/lib/collectors/network-monitor.ts
- packages/cli/src/commands/benchmark/metrics-calculator.ts
- packages/cli/src/lib/collectors/cookie-banner-collector.ts
- packages/cli/src/lib/performance.ts
- packages/cli/src/commands/benchmark/benchmark-runner.ts
- packages/cli/src/commands/benchmark/resource-collector.ts
- packages/cli/src/commands/benchmark/types.ts
- packages/cli/src/commands/benchmark/constants.ts
- packages/cli/tsconfig.json
- packages/cli/src/lib/collectors/resource-timing-collector.ts
- packages/cli/src/lib/README.md
- packages/cli/base.json
- packages/cli/src/types/index.ts
- packages/cli/package.json
- packages/cli/src/commands/benchmark/cookie-banner-detector.ts
- packages/cli/src/commands/results.ts
🧰 Additional context used
🧬 Code graph analysis (25)
benchmarks/with-onetrust/app/layout.tsx (5)
benchmarks/baseline/app/layout.tsx (1)
metadata(10-12)benchmarks/with-c15t-nextjs/app/layout.tsx (1)
metadata(16-18)benchmarks/with-c15t-react/app/layout.tsx (1)
metadata(16-18)benchmarks/with-iubenda/app/layout.tsx (1)
metadata(4-6)benchmarks/with-ketch/app/layout.tsx (1)
metadata(4-6)
packages/cookiebench-cli/src/index.ts (9)
packages/cookiebench-cli/src/utils/logger.ts (3)
logger(152-152)CliLogger(8-8)createCliLogger(102-149)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/cookiebench-cli/src/utils/constants.ts (1)
HALF_SECOND(2-2)packages/cookiebench-cli/src/components/intro.ts (1)
displayIntro(11-93)packages/cookiebench-cli/src/commands/benchmark.ts (1)
benchmarkCommand(317-501)packages/cookiebench-cli/src/commands/results.ts (1)
resultsCommand(884-1114)packages/cookiebench-cli/src/commands/scores.ts (1)
scoresCommand(93-183)packages/cookiebench-cli/src/commands/save.ts (1)
saveCommand(228-391)packages/cookiebench-cli/src/commands/db.ts (1)
dbCommand(46-125)
packages/benchmark/src/constants.ts (1)
packages/benchmark/src/index.ts (2)
BENCHMARK_CONSTANTS(5-5)BUNDLE_TYPES(5-5)
benchmarks/with-iubenda/app/layout.tsx (5)
benchmarks/baseline/app/layout.tsx (1)
metadata(10-12)benchmarks/with-c15t-nextjs/app/layout.tsx (1)
metadata(16-18)benchmarks/with-c15t-react/app/layout.tsx (1)
metadata(16-18)benchmarks/with-ketch/app/layout.tsx (1)
metadata(4-6)benchmarks/with-onetrust/app/layout.tsx (1)
metadata(4-6)
benchmarks/with-ketch/app/layout.tsx (5)
benchmarks/baseline/app/layout.tsx (1)
metadata(10-12)benchmarks/with-c15t-nextjs/app/layout.tsx (1)
metadata(16-18)benchmarks/with-c15t-react/app/layout.tsx (1)
metadata(16-18)benchmarks/with-iubenda/app/layout.tsx (1)
metadata(4-6)benchmarks/with-onetrust/app/layout.tsx (1)
metadata(4-6)
packages/benchmark/src/perfume-collector.ts (1)
packages/benchmark/src/types.ts (2)
WindowWithPerfumeMetrics(261-277)PerfumeMetrics(228-259)
benchmarks/baseline/app/layout.tsx (5)
benchmarks/with-c15t-nextjs/app/layout.tsx (1)
metadata(16-18)benchmarks/with-c15t-react/app/layout.tsx (1)
metadata(16-18)benchmarks/with-iubenda/app/layout.tsx (1)
metadata(4-6)benchmarks/with-ketch/app/layout.tsx (1)
metadata(4-6)benchmarks/with-onetrust/app/layout.tsx (1)
metadata(4-6)
packages/benchmark/src/resource-timing-collector.ts (2)
packages/benchmark/src/types.ts (1)
ResourceTimingData(126-211)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(1-13)
packages/benchmark/src/bundle-strategy.ts (2)
packages/benchmark/src/types.ts (2)
Config(13-56)BundleStrategy(119-123)packages/benchmark/src/constants.ts (1)
BUNDLE_TYPES(15-20)
packages/cookiebench-cli/src/commands/scores.ts (6)
packages/cookiebench-cli/src/commands/results.ts (1)
RawBenchmarkDetail(39-179)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/benchmark/src/types.ts (1)
Config(13-56)packages/cookiebench-cli/src/utils/constants.ts (2)
HALF_SECOND(2-2)PERCENTAGE_DIVISOR(6-6)packages/cookiebench-cli/src/utils/scoring.ts (2)
printScores(1402-1441)calculateScores(1029-1398)
packages/cookiebench-cli/src/commands/db.ts (3)
packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/runner/src/utils.ts (1)
ONE_SECOND(5-5)
packages/cookiebench-cli/src/utils/index.ts (2)
packages/runner/src/utils.ts (4)
readConfig(6-16)formatTime(18-23)ONE_SECOND(5-5)getPackageManager(25-57)packages/benchmark/src/types.ts (1)
Config(13-56)
packages/cookiebench-cli/src/commands/save.ts (6)
packages/cookiebench-cli/src/commands/results.ts (2)
BenchmarkOutput(181-215)RawBenchmarkDetail(39-179)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/cookiebench-cli/src/utils/constants.ts (2)
HALF_SECOND(2-2)PERCENTAGE_DIVISOR(6-6)packages/cookiebench-cli/src/utils/scoring.ts (1)
calculateScores(1029-1398)
benchmarks/with-c15t-react/app/layout.tsx (5)
benchmarks/baseline/app/layout.tsx (1)
metadata(10-12)benchmarks/with-c15t-nextjs/app/layout.tsx (1)
metadata(16-18)benchmarks/with-iubenda/app/layout.tsx (1)
metadata(4-6)benchmarks/with-ketch/app/layout.tsx (1)
metadata(4-6)benchmarks/with-onetrust/app/layout.tsx (1)
metadata(4-6)
packages/runner/src/benchmark-runner.ts (2)
packages/runner/src/types.ts (3)
Config(6-6)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/runner/src/performance-aggregator.ts (1)
PerformanceAggregator(29-372)
packages/benchmark/src/network-monitor.ts (2)
packages/benchmark/src/types.ts (3)
NetworkRequest(104-111)NetworkMetrics(113-116)Config(13-56)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(1-13)
packages/cookiebench-cli/src/commands/results.ts (6)
packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/benchmark/src/types.ts (1)
Config(13-56)packages/cookiebench-cli/src/types/index.ts (2)
Config(5-5)BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/constants.ts (18)
KILOBYTE(9-9)SCORE_THRESHOLD_POOR(13-13)SCORE_THRESHOLD_FAIR(14-14)CLS_DECIMAL_PLACES(10-10)PERCENTAGE_DIVISOR(6-6)CLS_THRESHOLD_GOOD(17-17)CLS_THRESHOLD_FAIR(18-18)COL_WIDTH_NAME(21-21)COL_WIDTH_CHART_PADDING(27-27)MAX_FILENAME_LENGTH(30-30)TRUNCATED_FILENAME_LENGTH(31-31)MIN_DURATION_THRESHOLD(34-34)COL_WIDTH_TYPE(22-22)COL_WIDTH_SOURCE(23-23)COL_WIDTH_SIZE(24-24)COL_WIDTH_DURATION(25-25)COL_WIDTH_TAGS(26-26)DEFAULT_DOM_SIZE(4-4)packages/cookiebench-cli/src/utils/scoring.ts (1)
calculateScores(1029-1398)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)
packages/benchmark/src/cookie-banner-collector.ts (3)
packages/benchmark/src/types.ts (5)
Config(13-56)CookieBannerMetrics(78-91)WindowWithCookieMetrics(65-76)LayoutShiftEntry(59-62)CookieBannerData(93-101)packages/benchmark/src/bundle-strategy.ts (1)
determineBundleStrategy(4-21)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(1-13)
packages/runner/src/types.ts (2)
packages/cookiebench-cli/src/types/index.ts (3)
ServerInfo(6-6)BenchmarkDetails(3-3)BenchmarkResult(4-4)packages/runner/src/index.ts (3)
ServerInfo(21-21)BenchmarkDetails(10-10)BenchmarkResult(11-11)
packages/runner/src/performance-aggregator.ts (2)
packages/benchmark/src/types.ts (8)
CoreWebVitals(214-225)CookieBannerData(93-101)CookieBannerMetrics(78-91)NetworkRequest(104-111)NetworkMetrics(113-116)ResourceTimingData(126-211)Config(13-56)PerfumeMetrics(228-259)packages/runner/src/types.ts (10)
CoreWebVitals(10-10)CookieBannerData(8-8)CookieBannerMetrics(9-9)NetworkRequest(12-12)NetworkMetrics(11-11)ResourceTimingData(14-14)Config(6-6)PerfumeMetrics(13-13)BenchmarkDetails(24-156)BenchmarkResult(158-271)
packages/cookiebench-cli/src/commands/benchmark.ts (7)
packages/runner/src/types.ts (2)
BenchmarkResult(158-271)ServerInfo(18-21)packages/cookiebench-cli/src/utils/constants.ts (6)
DEFAULT_THIRD_PARTY_DOMAINS(5-5)PERCENTAGE_DIVISOR(6-6)DEFAULT_DOM_SIZE(4-4)HALF_SECOND(2-2)DEFAULT_ITERATIONS(3-3)SEPARATOR_WIDTH(7-7)packages/runner/src/benchmark-runner.ts (2)
runSingleBenchmark(37-123)BenchmarkRunner(15-187)packages/runner/src/utils.ts (1)
readConfig(6-16)packages/runner/src/server.ts (2)
buildAndServeNextApp(6-71)cleanupServer(73-77)packages/cookiebench-cli/src/utils/scoring.ts (2)
calculateScores(1029-1398)printScores(1402-1441)packages/cookiebench-cli/src/commands/results.ts (1)
resultsCommand(884-1114)
packages/runner/src/server.ts (2)
packages/runner/src/types.ts (1)
ServerInfo(18-21)packages/runner/src/utils.ts (2)
getPackageManager(25-57)ONE_SECOND(5-5)
packages/runner/src/utils.ts (2)
packages/cookiebench-cli/src/utils/index.ts (3)
readConfig(10-19)formatTime(21-26)getPackageManager(28-60)packages/benchmark/src/types.ts (1)
Config(13-56)
packages/cookiebench-cli/src/utils/scoring.ts (2)
packages/runner/src/utils.ts (1)
ONE_SECOND(5-5)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)
packages/cookiebench-cli/src/components/intro.ts (1)
packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)
🪛 LanguageTool
packages/cookiebench-cli/README.md
[uncategorized] ~373-~373: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ice overhead 4. Transparency (15%): Open source status, documentation, licensing 5. **U...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🪛 markdownlint-cli2 (0.18.1)
packages/benchmark/README.md
99-99: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
99-99: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🔇 Additional comments (74)
turbo.json (1)
1-28: Workspace-level task definitions sufficiently cover all new packages.Verification confirms that @consentio/benchmark, @consentio/runner, and cookiebench-cli all define npm scripts (build, check-types, lint, fmt, dev) that align with the workspace-level turbo.json tasks. All build outputs target dist/ (covered by turbo.json), and static-only packages (benchmark-schema, typescript-config) have no scripts. The simplified configuration is appropriate and requires no custom package-level overrides.
benchmarks/with-cookie-yes/app/layout.tsx (1)
1-2: LGTM! Formatting standardization applied.The quote style changes (single → double quotes) are purely cosmetic and align with the PR's broader formatting standardization effort across benchmark configurations.
Also applies to: 5-5
benchmarks/with-ketch/next.config.ts (1)
1-1: LGTM! Quote style normalized.Import statement formatting updated to use double quotes, consistent with the formatting standardization across all benchmark configurations in this PR.
benchmarks/with-c15t-react/next.config.ts (1)
1-1: LGTM! Quote style normalized.Import statement formatting updated to use double quotes, consistent with the formatting standardization across all benchmark configurations in this PR.
benchmarks/with-iubenda/next.config.ts (1)
1-1: LGTM! Quote style normalized.Import statement formatting updated to use double quotes, consistent with the formatting standardization across all benchmark configurations in this PR.
benchmarks/with-enzuzo/next.config.ts (1)
1-1: LGTM! Quote style normalized.Import statement formatting updated to use double quotes, consistent with the formatting standardization across all benchmark configurations in this PR.
benchmarks/with-enzuzo/config.json (1)
1-40: LGTM! JSON formatting normalized.The changes are purely cosmetic indentation and whitespace adjustments with no semantic modifications to configuration values. This aligns with the PR's formatting standardization effort.
benchmarks/with-didomi/next.config.ts (1)
1-1: LGTM! Quote style normalized.Import statement formatting updated to use double quotes, consistent with the formatting standardization across all benchmark configurations in this PR.
benchmarks/with-usercentrics/config.json (1)
1-40: LGTM! JSON formatting normalized.The changes are purely cosmetic indentation and whitespace adjustments with no semantic modifications to configuration values. This aligns with the PR's formatting standardization effort.
benchmarks/with-c15t-react/config.json (1)
2-47: Schema reference requires package validation.All JSON benchmark configs now reference
./node_modules/@cookiebench/benchmark-schema/schema.json. Verify that the@cookiebench/benchmark-schemapackage is properly declared as a dependency and available to all benchmark packages during development and build time.benchmarks/with-didomi/config.json (1)
2-48: Formatting consistency maintained.Indentation and whitespace standardized; configuration semantics unchanged.
benchmarks/baseline/config.json (1)
2-29: Formatting consistency maintained.Indentation and whitespace standardized; configuration semantics unchanged.
benchmarks/with-osano/config.json (1)
2-39: Verify bundleType value consistency.Line 16 specifies
"bundleType": "iffe". Confirm this is the intended spelling—likely should be"iife"(Immediately Invoked Function Expression). Verify against the schema definition to ensure consistency with other bundle type values.benchmarks/with-onetrust/next.config.ts (1)
1-1: Quote style normalization applied.Import quote changed from single to double quotes, consistent with project-wide formatting updates.
benchmarks/with-ketch/config.json (1)
2-38: Verify bundleType value consistency.Line 19 specifies
"bundleType": "iffe", same as with-osano config. Confirm this spelling is intentional—expected value is typically"iife"(Immediately Invoked Function Expression). Verify against schema definition.benchmarks/with-onetrust/app/layout.tsx (1)
1-2: Quote style normalization applied.Import quotes and string literals updated from single to double quotes, consistent with project-wide formatting updates. No runtime behavior changes.
Also applies to: 23-23
packages/cookiebench-cli/rslib.config.ts (2)
1-24: Verify figlet exclusion rationale.The exclude list includes
"figlet"(line 8). Confirm this is intentional—whether figlet is an optional dependency, only imported conditionally in CLI commands, or should be externalized. If figlet is a significant dependency, consider documenting this decision in comments or a CHANGELOG.
3-24: RSLib configuration properly aligned with CLI requirements.Configuration mirrors packages/benchmark pattern: ESM format for modern Node.js, declarations enabled, bundle mode active, and clean dist output. Target set to "node" correctly identifies this as a CLI package.
benchmarks/baseline/next-env.d.ts (1)
3-3: Verify manual modification of auto-generated file.Line 5 states "This file should not be edited," yet this import is being added manually. Ensure this change aligns with Next.js best practices for route type augmentation, or confirm if this should be auto-generated by Next.js tooling instead.
benchmarks/baseline/next.config.ts (1)
1-1: LGTM!Formatting change standardizes quote style across benchmark configurations.
benchmarks/with-cookie-yes/config.json (1)
1-44: LGTM!Formatting standardization improves consistency across benchmark configurations without changing any semantic content.
benchmarks/with-c15t-react/tsconfig.json (1)
2-10: LGTM!Formatting standardization without functional changes.
packages/benchmark/tsconfig.json (1)
1-15: TypeScript version requirement verified—no issues found.The
packages/benchmark/package.jsonspecifies TypeScript^5.9.3, which satisfies the minimum TypeScript 5.0+ requirement formoduleResolution: "bundler". The configuration is correct.benchmarks/with-enzuzo/app/layout.tsx (1)
1-19: LGTM! Formatting standardization.The changes consistently update string literals from single to double quotes across imports and metadata, aligning with the project-wide formatting standards applied throughout this PR.
benchmarks/with-ketch/app/layout.tsx (1)
1-5: LGTM! Formatting standardization.Quote style updates align with the broader formatting standardization across benchmark files in this PR.
benchmarks/with-onetrust/config.json (1)
1-46: LGTM! Formatting normalization.The changes are purely formatting and whitespace adjustments with no semantic modifications to the configuration.
benchmarks/baseline/app/layout.tsx (1)
1-11: LGTM! Formatting standardization.Quote style updates are consistent with the formatting standardization across all benchmark layout files in this PR.
tsconfig.json (2)
4-8: LGTM! Improved readability.Reformatting the
lib,include, andexcludearrays to multi-line format improves readability with no semantic changes.Also applies to: 23-29
21-21: Manual TypeScript compilation verification required.The sandbox environment lacks TypeScript installation to execute the
tsc --noEmitverification. The codebase contains 30 TypeScript source files and already hasstrictNullChecks: trueenabled in tsconfig.json with TypeScript 5.9.3 as a dependency.To verify no type errors are introduced by this compiler option change, run
npm installfollowed bynpx tsc --noEmitin your local development environment or CI pipeline. This is essential before merging, as strictNullChecks is a significant compiler setting that enforces stricter null/undefined type checking.benchmarks/with-c15t-react/app/layout.tsx (1)
1-31: LGTM! Formatting standardization.String literal updates from single to double quotes align with the project-wide formatting standards.
packages/runner/rslib.config.ts (1)
1-14: LGTM! Standard build configuration.The rslib configuration is appropriate for a Node.js library package:
- ESM format for modern module compatibility
- ES2021 syntax for current JavaScript features
- Type declaration generation enabled
- Node target correctly specified
This aligns with similar configurations across the monorepo (e.g., packages/benchmark/rslib.config.ts).
packages/benchmark/rslib.config.ts (1)
1-14: LGTM!The RSlib configuration is appropriate for a library package with ESM output targeting Node.js. The TypeScript declaration generation (
dts: true) ensures proper type support for consumers of this package..cursor/rules/ultracite.mdc (1)
1-5: Verify thealwaysApply: falsesetting.The ultracite rules are configured with
alwaysApply: false, meaning they won't be automatically enforced. Is this intentional, or should these rules be always applied for consistent code quality across the project?If these rules should be enforced by default, apply this diff:
--- description: Ultracite Rules - AI-Ready Formatter and Linter globs: "**/*.{ts,tsx,js,jsx,json,jsonc,html,vue,svelte,astro,css,yaml,yml,graphql,gql,md,mdx,grit}" -alwaysApply: false +alwaysApply: true ---benchmarks/with-iubenda/app/layout.tsx (1)
1-6: LGTM!The formatting changes standardize quote styles across the codebase. No functional changes detected.
packages/cookiebench-cli/base.json (1)
9-9: Verify the need for DOM libraries in a CLI package.The CLI configuration includes
"DOM"and"DOM.Iterable"in the lib array, which are typically used for browser environments. CLI tools usually only need Node.js types. Is there browser-related code in this package, or can these be removed?If DOM types aren't needed, apply this diff:
- "lib": ["es2022", "DOM", "DOM.Iterable"], + "lib": ["es2022"],Otherwise, please clarify why DOM types are required for the CLI package.
packages/runner/tsconfig.json (1)
1-15: LGTM!The TypeScript configuration is well-structured for a library package with modern module resolution and declaration generation enabled.
packages/cookiebench-cli/README.md (1)
1-395: Excellent documentation!The README is comprehensive, well-organized, and provides clear examples for all commands. The distinction between admin-only and public commands is particularly helpful, and the interactive mode documentation with examples greatly improves usability.
biome.jsonc (1)
1-21: LGTM: Clean configuration simplificationThe migration to ultracite presets and local schema reference is well-structured. The simplified rules section indicates that most configuration is now handled by the extended presets, which improves maintainability.
package.json (2)
5-12: LGTM: Clean script reorganizationThe new scripts align well with the CLI refactoring. The migration from
@cookiebench/clitocookiebenchand the addition ofcheck-typesanddbcommands improve the developer experience.
14-28: All specified dependency versions are current.The verification confirms that each dependency is at its latest published version:
- @playwright/test, typescript, turbo, and @biomejs/biome all match their latest npm versions
- No outdated or stale versions present
The code requires no changes.
benchmarks/with-cookie-yes/package.json (1)
1-25: LGTM: Clean Next.js 16 and React 19 upgradeThe dependency updates and script simplification align well with the PR's refactoring goals. The explicit port specification (
--port 3001) is useful for running multiple benchmarks in parallel during development.packages/benchmark/package.json (1)
1-33: LGTM: Well-structured package manifestThe package configuration follows modern Node.js/TypeScript best practices with clear ESM exports, type definitions, and appropriate tooling. The dependency choices (Playwright for browser automation, perfume.js for metrics) align well with the benchmark package's purpose.
packages/runner/package.json (1)
1-33: LGTM: Consistent package structureThe runner package follows the same structure as the benchmark package, maintaining consistency across the monorepo. The workspace dependency on
@consentio/benchmarkis correctly specified.packages/cookiebench-cli/src/components/intro.ts (1)
11-40: LGTM: Good error handling for figlet generationThe async figlet generation with proper error handling and fallback ensures the CLI remains functional even if the ASCII art generation fails.
benchmarks/with-ketch/package.json (1)
1-24: LGTM: Consistent benchmark package upgradeThe updates mirror the pattern used in other benchmark packages, maintaining consistency across the monorepo. Note that this package uses the default Next.js dev port (3000), while
with-cookie-yesexplicitly uses port 3001.packages/benchmark/src/bundle-strategy.ts (1)
4-21: Edge-case handling is correct; no issues foundThe function properly handles the mentioned edge cases:
- undefined bundleType: Both
isIIFEandisBundledcorrectly evaluate tofalse(all equality and Array.isArray checks fail)- empty array
[]: Both flags correctly evaluate tofalse(includes() returns false for empty arrays)The caller in
cookie-banner-collector.tsgracefully handles this by defaultingbundleStrategyto "Unknown" when both flags are false. The defensive programming pattern (optional chaining despitetechStackbeing required in the type) is intentional and safe.packages/benchmark/src/constants.ts (1)
1-13: LGTM! Well-structured benchmark constants.The timing and polling constants are appropriately defined with clear comments and reasonable values. Using
as constensures type safety and immutability.benchmarks/with-onetrust/package.json (1)
2-24: LGTM! Standard Next.js 16 upgrade.The package.json updates align with the PR's migration to Next.js 16, React 19.2, and simplified tooling. The port assignment (3006) is noted for conflict checking.
benchmarks/with-osano/app/layout.tsx (1)
1-21: LGTM! Formatting standardization.The changes only update quote style from single to double quotes, aligning with the project's formatting standards. No functional changes.
packages/cookiebench-cli/src/utils/logger.ts (1)
102-149: LGTM! Well-structured CLI logger implementation.The logger properly extends the base logger with CLI-specific methods (message, note, step, outro, clear) and integrates well with clack prompts. The design allows for flexible log level handling while maintaining type safety.
packages/benchmark/src/cookie-banner-collector.ts (1)
25-58: LGTM! Clean initialization logic.The bundle strategy detection and metrics initialization are well-structured. The debug logging provides good visibility into the detected configuration.
packages/benchmark/src/types.ts (1)
1-277: LGTM! Comprehensive type definitions.The type definitions are well-structured, thoroughly documented, and provide strong type safety for the benchmark module. The separation of concerns (config types, performance types, network types, etc.) makes the types easy to understand and maintain.
benchmarks/with-c15t-nextjs/next.config.ts (1)
1-7: LGTM! Formatting standardization.Quote style updated for consistency across the codebase.
benchmarks/with-onetrust/tsconfig.json (1)
1-11: LGTM! Formatting standardization.Indentation and whitespace formatting updated for consistency.
benchmarks/baseline/tsconfig.json (1)
1-11: LGTM! Formatting standardization.Indentation and whitespace formatting updated for consistency.
benchmarks/with-iubenda/tsconfig.json (1)
1-11: LGTM! Formatting standardization.Indentation and whitespace formatting updated for consistency.
benchmarks/with-usercentrics/tsconfig.json (1)
1-11: LGTM! Formatting standardization.Indentation and whitespace formatting updated for consistency.
benchmarks/baseline/package.json (1)
12-25: Verification complete: all package versions are stable and secure.The installed versions match the latest available releases and are not affected by the reported security advisories. The vulnerabilities listed in the advisory database target older major versions (Next.js < 16, React < 19, React DOM < 19), which are not in use. No action required.
benchmarks/with-c15t-nextjs/config.json (1)
2-47: Formatting update is clean.The JSON reformatting maintains structural and semantic integrity. All keys and values remain unchanged.
benchmarks/with-cookie-yes/tsconfig.json (1)
2-10: TypeScript configuration formatting is consistent.Indentation normalized without altering extends, compilerOptions, or paths configuration.
benchmarks/with-osano/tsconfig.json (1)
2-10: TypeScript configuration formatting applied consistently.Standard Next.js configuration structure maintained across benchmarks.
benchmarks/with-usercentrics/next.config.ts (1)
1-1: Import quote normalization applied.Double-quote import style now aligns with standardized formatting across benchmark configs.
benchmarks/with-enzuzo/tsconfig.json (1)
2-10: TypeScript configuration formatting standardized.Indentation adjustments applied without altering configuration semantics.
benchmarks/with-c15t-nextjs/tsconfig.json (1)
2-10: TypeScript configuration formatting consistent with broader repository updates.Formatting standardization applied without altering configuration values.
benchmarks/with-cookie-control/next.config.ts (1)
1-1: Import quote normalization applied.Double-quote import style standardized across benchmark Next.js configurations.
benchmarks/with-cookie-control/config.json (1)
2-39: Configuration JSON formatting standardized.Indentation and spacing updated with all keys and values preserved intact.
benchmarks/with-usercentrics/package.json (1)
1-25: LGTM! Clean dependency upgrades and script simplification.The upgrade to Next.js 16, React 19.2, and TypeScript 5.9 with simplified scripts and proper workspace dependencies looks good.
packages/runner/README.md (1)
1-150: Excellent documentation for the new runner package.The README provides comprehensive coverage of the runner's features, clear usage examples, and a helpful configuration schema. This will make the package easy to adopt and use.
packages/cookiebench-cli/src/types/index.ts (1)
1-7: LGTM! Clean type re-exports.Re-exporting types from the runner package provides a convenient interface for CLI consumers.
packages/cookiebench-cli/src/index.ts (3)
1-24: LGTM! Clean initialization and admin gating.The logger initialization, admin check, and cancellation handler are well-implemented. The use of environment-based log level configuration is a good practice.
26-75: Well-structured command routing with proper admin gating.The direct command execution path correctly gates admin commands and provides helpful error messages with available commands.
76-129: Interactive mode provides good UX.The interactive prompt mode with conditional admin options provides a clear user experience.
packages/runner/src/index.ts (1)
3-24: Public surface looks coherent.
Exports align with the reorganized runner modules, keeping consumption straightforward.packages/benchmark/src/index.ts (1)
3-26: Benchmark barrel looks consistent.
The export surface neatly mirrors the new module layout and keeps consumers on a single import path.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 12
♻️ Duplicate comments (30)
benchmarks/baseline/next-env.d.ts (1)
3-3: Same issue: Manual modification of auto-generated file.This file has the same issue as
benchmarks/with-c15t-nextjs/next-env.d.ts. The manual import of a build-time artifact contradicts the file's explicit guidance not to edit it and may cause TypeScript errors before the first build.#!/bin/bash # Description: Check Next.js config and verify the pattern across all benchmark directories echo "=== Checking Next.js config for typed routes configuration ===" fd -e js -e ts -e mjs 'next.config' benchmarks/baseline/ --exec cat {} echo -e "\n=== Searching for all next-env.d.ts files with this import pattern ===" rg -n "import.*\.next/types/routes" benchmarks/.gitignore (1)
46-139: Duplicate: Remove unnecessary .gitignore entries.This issue was already flagged in a previous review. The 94 entries for
c15t/cli/andc15t/logger/reference non-existent directories and should be removed.benchmarks/with-iubenda/package.json (1)
6-6: Port conflict still unresolved.As noted in the previous review, port 3001 conflicts with multiple other benchmark packages (
with-c15t-nextjs,with-cookie-control,with-cookie-yes,with-enzuzo,with-usercentrics), preventing simultaneous execution during benchmark comparisons.benchmarks/with-osano/package.json (1)
6-6: Port conflict with benchmarks/with-onetrust remains unresolved.As flagged in the previous review, both
with-osanoandwith-onetrustuse port 3006, preventing simultaneous benchmark execution.packages/cookiebench-cli/src/types/index.ts (1)
12-34: Align grade/status casing.The top-level
gradeuses Title Case strings, butstatusvalues stay lowercase. This mismatch forces consumers to normalize casing in order to compare them. Please pick one convention (e.g., Title Case everywhere) and update both unions so they stay consistent.packages/cookiebench-cli/package.json (1)
21-30: Avoid shipping on an alpha of@clack/prompts.
^1.0.0-alpha.0is still pre-release and has churn risks. Please stick to the latest stable (currently0.11.0) until v1 is officially released, or document why alpha stability is acceptable for production use.packages/benchmark/src/constants.ts (1)
15-19: Fix theIIFEtypo.
BUNDLE_TYPES.IIFEshould be"iife"—the current"iffe"value will cause bundle strategy checks to fail. Correct the string so downstream comparisons work.export const BUNDLE_TYPES = { - IIFE: "iffe", + IIFE: "iife", ESM: "esm", CJS: "cjs", BUNDLED: "bundled", } as const;benchmarks/with-enzuzo/package.json (1)
6-6: Port conflict remains unresolved.As previously noted, both
with-enzuzoandwith-iubendause port 3001, preventing simultaneous execution during benchmark comparisons.benchmarks/with-cookie-control/package.json (1)
12-14: Node.js version requirement remains unaddressed.As previously noted, Next.js 16.0.1 and React 19.2.0 require Node >= 20.9.0. Without an explicit
enginesfield, environments on older Node versions will fail to build or run.packages/cookiebench-cli/README.md (1)
99-113: Terminal compatibility note for checkbox glyphs.As previously noted, the checkbox glyphs (◼) used in the interactive example may not render consistently across all terminals.
packages/cookiebench-cli/src/utils/auth.ts (1)
5-10: Case-sensitive admin check will reject uppercase values.The current implementation only accepts lowercase strings ("true", "1", "yes"), but environment variables are commonly set as uppercase literals (e.g.,
CONSENT_ADMIN=TRUE). This will prevent legitimate admin users from accessing admin commands.packages/runner/src/utils.ts (3)
6-16: Inconsistent error handling across packages.The
readConfigfunction logs errors to console (line 13), while the identical function inpackages/cookiebench-cli/src/utils/index.tssilently returns null with the expectation that the caller will log. This inconsistency was previously identified and should be resolved by aligning both implementations.
25-57: Code duplication and complex nesting require refactoring.This function duplicates logic from
packages/cookiebench-cli/src/utils/index.ts(lines 28-60) and uses deeply nested try-catch blocks (3 levels). The duplication and nesting issues were previously identified and should be addressed by consolidating into a single shared implementation with flattened structure.
55-56: Remove unreachable return statement.Line 56 is unreachable because line 51 always returns in the final catch block. This dead code was previously identified and should be removed.
packages/cookiebench-cli/src/index.ts (1)
114-128: Consider adding default case for defensive programming.The biome-ignore directive on line 114 bypasses the linter's requirement for a default case. While the
select()prompt (lines 105-108) constrains possible values, adding a default case was previously suggested as defensive programming practice and would eliminate the need for the ignore directive.packages/cookiebench-cli/src/utils/index.ts (1)
10-60: Deduplicate CLI utilities with the runner package
readConfig,formatTime, andgetPackageManagerare copied from@consentio/runner/src/utils.ts, but the copies have already drifted (noconfigPathsupport, different logging/error semantics). This duplication guarantees the two surfaces will diverge further and forces us to patch bugs twice. Please re-export the runner implementations instead of maintaining a fork.-import { readFileSync } from "node:fs"; -import { join } from "node:path"; -import type { Config } from "../types"; -import { ONE_SECOND } from "./constants"; +import type { Config } from "../types"; +import { + formatTime as runnerFormatTime, + getPackageManager as runnerGetPackageManager, + readConfig as runnerReadConfig, +} from "@consentio/runner"; +import { ONE_SECOND } from "./constants"; export * from "./constants"; -export function readConfig(): Config | null { - try { - const configPath = join(process.cwd(), "config.json"); - const configContent = readFileSync(configPath, "utf-8"); - return JSON.parse(configContent) as Config; - } catch { - return null; - } -} - -export function formatTime(ms: number): string { - if (ms < ONE_SECOND) { - return `${ms.toFixed(0)}ms`; - } - return `${(ms / ONE_SECOND).toFixed(2)}s`; -} - -export async function getPackageManager(): Promise<{ - command: string; - args: string[]; -}> { - // ... -} +export const readConfig = (configPath?: string): Config | null => + (runnerReadConfig(configPath) as Config | null); +export const formatTime = runnerFormatTime; +export const getPackageManager = runnerGetPackageManager;This keeps the CLI aligned with the runner’s behavior and avoids another maintenance hot spot.
packages/runner/src/server.ts (1)
55-70: Ensure we clean up the Next.js process when startup fails.If all health checks fail we throw, but the spawned Next.js process keeps running and keeps the port bound. We need to terminate it (and wait briefly for exit) before propagating the error to avoid leaking processes.
- throw new Error("Server failed to start"); + const startupError = new Error("Server failed to start"); + if (!serverProcess.killed) { + await new Promise<void>((resolve) => { + const timeout = setTimeout(resolve, ONE_SECOND * 2); + serverProcess.once("exit", () => { + clearTimeout(timeout); + resolve(); + }); + serverProcess.kill("SIGTERM"); + }); + } + throw startupError;packages/benchmark/src/resource-timing-collector.ts (2)
88-111: Fix third-party detection to avoid substring false positives.Using
entry.name.includes(window.location.hostname)misclassifies hosts likehttps://notexample.comas first-party because the hostname happens to contain ours. Parse the URL and compare the hostname instead (fall back to origin/relative checks on failures) before using it in all bundled/third-party filters and resource flags.+ const isFirstParty = (entry: PerformanceResourceTiming) => { + try { + const entryUrl = new URL(entry.name, window.location.origin); + return entryUrl.hostname === window.location.hostname; + } catch { + return ( + entry.name.startsWith(window.location.origin) || + entry.name.startsWith("/") + ); + } + }; ... - bundled: calculateSize( - scriptEntries.filter((e) => - e.name.includes(window.location.hostname) - ) - ), - thirdParty: calculateSize( - scriptEntries.filter( - (e) => !e.name.includes(window.location.hostname) - ) - ), + bundled: calculateSize( + scriptEntries.filter((entry) => isFirstParty(entry)) + ), + thirdParty: calculateSize( + scriptEntries.filter((entry) => !isFirstParty(entry)) + ), ... - thirdParty: calculateSize( - scriptEntries.filter( - (e) => !e.name.includes(window.location.hostname) - ) - ), + thirdParty: calculateSize( + scriptEntries.filter((entry) => !isFirstParty(entry)) + ), ... - isThirdParty: !entry.name.includes(window.location.hostname), + isThirdParty: !isFirstParty(entry),Also applies to: 119-162
59-66: Report the actual document language instead of hardcoding"en".Every non-English benchmark is currently labeled English, skewing downstream analysis. Read the
<html lang>attribute, fall back tonavigator.language/navigator.languages, and only then default to"en".- return { + const docLang = + (document.documentElement.getAttribute("lang") || "").trim(); + const language = + docLang || + navigator.language || + (Array.isArray(navigator.languages) && navigator.languages.length > 0 + ? navigator.languages[0] + : "en"); + + return { ... - language: "en", + language,Also applies to: 164-166
packages/cookiebench-cli/src/commands/db.ts (1)
30-44: Make drizzle-kit invocation path-safe.This issue was flagged in a previous review. Building the shell command with
cd ${DB_PACKAGE_PATH}breaks when the repository path contains spaces (e.g.,C:\Users\Jane Doe\...). Use thecwdoption instead to make the invocation portable across all platforms.Apply this diff:
function runDrizzleCommand(logger: CliLogger, command: string): void { try { logger.step(`Running: ${color.cyan(`drizzle-kit ${command}`)}`); - execSync(`cd ${DB_PACKAGE_PATH} && pnpm drizzle-kit ${command}`, { + execSync(`pnpm drizzle-kit ${command}`, { stdio: "inherit", encoding: "utf-8", + cwd: DB_PACKAGE_PATH, }); } catch (error) {packages/benchmark/src/perfume-collector.ts (4)
15-70: Security and reliability concerns with CDN script injection.This issue was flagged in a previous review. Several concerns with loading Perfume.js from a CDN without Subresource Integrity (SRI) protection, hardcoded version, silent failure on load errors, and external dependency on unpkg.com availability. Consider bundling Perfume.js with the benchmark package or adding SRI hash validation.
78-78: Fixed timeout may not work for all network conditions.This issue was flagged in a previous review. The hardcoded
PERFUME_METRICS_WAITtimeout might be insufficient for slow networks or heavily loaded pages. Consider making the timeout configurable or implementing a more sophisticated waiting strategy.
89-104: Using deprecated Performance Timing API.This issue was flagged in a previous review. The
performance.timingAPI is deprecated and will be removed from browsers. Replace with the Performance Timeline API usingperformance.getEntriesByType('navigation')[0]cast toPerformanceNavigationTiming.
107-119: Handle experimental API more defensively.This issue was flagged in a previous review. The
navigator.connectionAPI is experimental and not supported in all browsers (notably Safari). Add more defensive checks and default values to prevent failures in unsupported environments.packages/cookiebench-cli/src/commands/save.ts (2)
467-472: Count third-party requests across all asset types.This issue was flagged in a previous review. The
thirdPartyRequestscalculation only inspects scripts, missing any third-party CSS, images, fonts, or other resources. This underreports network impact and leads to incorrect scores.Apply this diff:
thirdPartyRequests: appResults.reduce( (a, b) => - a + b.resources.scripts.filter((s) => s.isThirdParty).length, + a + + b.resources.scripts.filter((r) => r.isThirdParty).length + + b.resources.styles.filter((r) => r.isThirdParty).length + + b.resources.images.filter((r) => r.isThirdParty).length + + b.resources.fonts.filter((r) => r.isThirdParty).length + + b.resources.other.filter((r) => r.isThirdParty).length, 0 ) / appResults.length,
544-550: Populate average payload metrics instead of zeroing them.This issue was flagged in a previous review.
scriptLoadTimeandscriptSizeare set to zero, andresourceCountonly counts scripts. This ships inaccurate data to the API. Calculate proper averages from the raw measurements inappResults.packages/benchmark/src/network-monitor.ts (1)
33-56: Fix third-party detection logic.This issue was flagged in a previous review. Line 34 sets
isThirdParty = !url.includes(new URL(url).hostname), but a URL always contains its own hostname, so this predicate is always false. This causes all requests to be classified as first-party, breaking network impact metrics and downstream scoring.packages/benchmark/src/cookie-banner-collector.ts (2)
144-146: Consider logging selector errors for debugging.This issue was flagged in a previous review. The empty catch block silently ignores selector errors. Consider logging these errors at debug level to aid troubleshooting invalid selectors in configurations.
214-217: Add detection check for hydration time consistency.This issue was flagged in a previous review. The
bannerHydrationTimeis calculated wheneverbannerInteractive > 0, but doesn't verify that the banner was actually detected. For consistency with other metrics, checkmetrics.detectedas well.packages/cookiebench-cli/src/commands/results.ts (1)
1044-1049: Fix third-party request counting.This issue was flagged in a previous review.
thirdPartyRequestsonly counts script resources, ignoring any third-party styles, images, fonts, or other assets. This underreports network impact and leads to incorrect scores.Apply this diff:
thirdPartyRequests: appResults.reduce( (a, b) => - a + b.resources.scripts.filter((s) => s.isThirdParty).length, + a + + b.resources.scripts.filter((r) => r.isThirdParty).length + + b.resources.styles.filter((r) => r.isThirdParty).length + + b.resources.images.filter((r) => r.isThirdParty).length + + b.resources.fonts.filter((r) => r.isThirdParty).length + + b.resources.other.filter((r) => r.isThirdParty).length, 0 ) / appResults.length,
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (94)
.cursor/rules/ultracite.mdc(1 hunks).gitignore(1 hunks)benchmarks/baseline/next-env.d.ts(1 hunks)benchmarks/baseline/package.json(2 hunks)benchmarks/with-c15t-nextjs/next-env.d.ts(1 hunks)benchmarks/with-c15t-nextjs/package.json(1 hunks)benchmarks/with-c15t-react/package.json(1 hunks)benchmarks/with-cookie-control/app/layout.tsx(1 hunks)benchmarks/with-cookie-control/config.json(1 hunks)benchmarks/with-cookie-control/next.config.ts(1 hunks)benchmarks/with-cookie-control/package.json(1 hunks)benchmarks/with-cookie-control/tsconfig.json(1 hunks)benchmarks/with-cookie-yes/package.json(1 hunks)benchmarks/with-didomi/package.json(0 hunks)benchmarks/with-enzuzo/app/layout.tsx(2 hunks)benchmarks/with-enzuzo/config.json(1 hunks)benchmarks/with-enzuzo/next.config.ts(1 hunks)benchmarks/with-enzuzo/package.json(1 hunks)benchmarks/with-enzuzo/tsconfig.json(1 hunks)benchmarks/with-iubenda/package.json(1 hunks)benchmarks/with-ketch/package.json(0 hunks)benchmarks/with-onetrust/package.json(1 hunks)benchmarks/with-osano/package.json(1 hunks)benchmarks/with-usercentrics/package.json(1 hunks)package.json(1 hunks)packages/benchmark/README.md(1 hunks)packages/benchmark/package.json(1 hunks)packages/benchmark/rslib.config.ts(1 hunks)packages/benchmark/src/bundle-strategy.ts(1 hunks)packages/benchmark/src/constants.ts(1 hunks)packages/benchmark/src/cookie-banner-collector.ts(1 hunks)packages/benchmark/src/index.ts(1 hunks)packages/benchmark/src/network-monitor.ts(1 hunks)packages/benchmark/src/perfume-collector.ts(1 hunks)packages/benchmark/src/resource-timing-collector.ts(1 hunks)packages/benchmark/src/types.ts(1 hunks)packages/benchmark/tsconfig.json(1 hunks)packages/cli/base.json(0 hunks)packages/cli/src/commands/benchmark/benchmark-runner.ts(0 hunks)packages/cli/src/commands/benchmark/bundle-strategy.ts(0 hunks)packages/cli/src/commands/benchmark/constants.ts(0 hunks)packages/cli/src/commands/benchmark/cookie-banner-detector.ts(0 hunks)packages/cli/src/commands/benchmark/index.ts(0 hunks)packages/cli/src/commands/benchmark/metrics-calculator.ts(0 hunks)packages/cli/src/commands/benchmark/network-monitor.ts(0 hunks)packages/cli/src/commands/benchmark/resource-collector.ts(0 hunks)packages/cli/src/commands/benchmark/types.ts(0 hunks)packages/cli/src/commands/db.ts(0 hunks)packages/cli/src/commands/results.ts(0 hunks)packages/cli/src/index.ts(0 hunks)packages/cli/src/lib/README.md(0 hunks)packages/cli/src/lib/benchmark-runner.ts(0 hunks)packages/cli/src/lib/collectors/cookie-banner-collector.ts(0 hunks)packages/cli/src/lib/collectors/index.ts(0 hunks)packages/cli/src/lib/collectors/network-monitor.ts(0 hunks)packages/cli/src/lib/collectors/resource-timing-collector.ts(0 hunks)packages/cli/src/lib/metrics/index.ts(0 hunks)packages/cli/src/lib/metrics/performance-aggregator.ts(0 hunks)packages/cli/src/lib/performance-enhanced.ts(0 hunks)packages/cli/src/lib/performance.ts(0 hunks)packages/cli/src/lib/server.ts(0 hunks)packages/cli/src/types/index.ts(0 hunks)packages/cli/src/utils/index.ts(0 hunks)packages/cli/tsconfig.json(0 hunks)packages/cookiebench-cli/README.md(1 hunks)packages/cookiebench-cli/base.json(1 hunks)packages/cookiebench-cli/package.json(1 hunks)packages/cookiebench-cli/rslib.config.ts(1 hunks)packages/cookiebench-cli/src/commands/benchmark.ts(1 hunks)packages/cookiebench-cli/src/commands/db.ts(1 hunks)packages/cookiebench-cli/src/commands/results.ts(1 hunks)packages/cookiebench-cli/src/commands/save.ts(1 hunks)packages/cookiebench-cli/src/commands/scores.ts(1 hunks)packages/cookiebench-cli/src/components/intro.ts(1 hunks)packages/cookiebench-cli/src/index.ts(1 hunks)packages/cookiebench-cli/src/types/index.ts(1 hunks)packages/cookiebench-cli/src/utils/auth.ts(1 hunks)packages/cookiebench-cli/src/utils/constants.ts(1 hunks)packages/cookiebench-cli/src/utils/index.ts(1 hunks)packages/cookiebench-cli/src/utils/logger.ts(1 hunks)packages/cookiebench-cli/src/utils/scoring.ts(27 hunks)packages/cookiebench-cli/tsconfig.json(1 hunks)packages/runner/README.md(1 hunks)packages/runner/package.json(1 hunks)packages/runner/rslib.config.ts(1 hunks)packages/runner/src/benchmark-runner.ts(1 hunks)packages/runner/src/index.ts(1 hunks)packages/runner/src/performance-aggregator.ts(1 hunks)packages/runner/src/server.ts(1 hunks)packages/runner/src/types.ts(1 hunks)packages/runner/src/utils.ts(1 hunks)packages/runner/tsconfig.json(1 hunks)tsconfig.json(2 hunks)turbo.json(1 hunks)
💤 Files with no reviewable changes (29)
- benchmarks/with-ketch/package.json
- packages/cli/src/index.ts
- packages/cli/src/lib/server.ts
- packages/cli/src/lib/README.md
- packages/cli/src/commands/benchmark/constants.ts
- packages/cli/src/lib/metrics/index.ts
- packages/cli/src/lib/collectors/index.ts
- benchmarks/with-didomi/package.json
- packages/cli/src/commands/benchmark/benchmark-runner.ts
- packages/cli/src/commands/benchmark/bundle-strategy.ts
- packages/cli/src/commands/db.ts
- packages/cli/base.json
- packages/cli/src/lib/performance-enhanced.ts
- packages/cli/src/lib/collectors/cookie-banner-collector.ts
- packages/cli/src/commands/benchmark/index.ts
- packages/cli/src/lib/metrics/performance-aggregator.ts
- packages/cli/src/commands/benchmark/metrics-calculator.ts
- packages/cli/src/utils/index.ts
- packages/cli/src/commands/benchmark/resource-collector.ts
- packages/cli/src/lib/benchmark-runner.ts
- packages/cli/src/lib/performance.ts
- packages/cli/src/commands/results.ts
- packages/cli/src/lib/collectors/resource-timing-collector.ts
- packages/cli/src/commands/benchmark/network-monitor.ts
- packages/cli/src/lib/collectors/network-monitor.ts
- packages/cli/src/commands/benchmark/types.ts
- packages/cli/src/types/index.ts
- packages/cli/tsconfig.json
- packages/cli/src/commands/benchmark/cookie-banner-detector.ts
🧰 Additional context used
🧬 Code graph analysis (20)
packages/benchmark/src/bundle-strategy.ts (3)
packages/benchmark/src/types.ts (2)
Config(13-56)BundleStrategy(119-123)packages/benchmark/src/constants.ts (1)
BUNDLE_TYPES(15-20)packages/cli/src/commands/benchmark/bundle-strategy.ts (1)
determineBundleStrategy(5-22)
packages/runner/src/server.ts (2)
packages/runner/src/types.ts (1)
ServerInfo(18-21)packages/runner/src/utils.ts (2)
getPackageManager(25-57)ONE_SECOND(5-5)
packages/cookiebench-cli/src/components/intro.ts (1)
packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)
packages/cookiebench-cli/src/commands/scores.ts (6)
packages/cookiebench-cli/src/commands/results.ts (1)
RawBenchmarkDetail(39-179)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/benchmark/src/types.ts (1)
Config(13-56)packages/cookiebench-cli/src/utils/constants.ts (2)
HALF_SECOND(2-2)PERCENTAGE_DIVISOR(6-6)packages/cookiebench-cli/src/utils/scoring.ts (2)
printScores(1402-1441)calculateScores(1029-1398)
packages/benchmark/src/network-monitor.ts (2)
packages/benchmark/src/types.ts (3)
NetworkRequest(104-111)NetworkMetrics(113-116)Config(13-56)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(1-13)
packages/benchmark/src/constants.ts (1)
packages/benchmark/src/index.ts (2)
BENCHMARK_CONSTANTS(5-5)BUNDLE_TYPES(5-5)
packages/cookiebench-cli/src/commands/save.ts (6)
packages/cookiebench-cli/src/commands/results.ts (2)
BenchmarkOutput(181-215)RawBenchmarkDetail(39-179)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/cookiebench-cli/src/utils/constants.ts (2)
HALF_SECOND(2-2)PERCENTAGE_DIVISOR(6-6)packages/cookiebench-cli/src/utils/scoring.ts (1)
calculateScores(1029-1398)
packages/cookiebench-cli/src/utils/index.ts (2)
packages/runner/src/utils.ts (4)
readConfig(6-16)formatTime(18-23)ONE_SECOND(5-5)getPackageManager(25-57)packages/benchmark/src/types.ts (1)
Config(13-56)
packages/runner/src/utils.ts (2)
packages/cookiebench-cli/src/utils/index.ts (3)
readConfig(10-19)formatTime(21-26)getPackageManager(28-60)packages/benchmark/src/types.ts (1)
Config(13-56)
packages/cookiebench-cli/src/index.ts (4)
packages/cookiebench-cli/src/utils/logger.ts (3)
logger(152-152)CliLogger(8-8)createCliLogger(102-149)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/cookiebench-cli/src/utils/constants.ts (1)
HALF_SECOND(2-2)packages/cookiebench-cli/src/components/intro.ts (1)
displayIntro(11-93)
packages/benchmark/src/resource-timing-collector.ts (2)
packages/benchmark/src/types.ts (1)
ResourceTimingData(126-211)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(1-13)
packages/runner/src/benchmark-runner.ts (8)
packages/benchmark/src/types.ts (1)
Config(13-56)packages/runner/src/types.ts (3)
Config(6-6)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/benchmark/src/cookie-banner-collector.ts (1)
CookieBannerCollector(13-242)packages/benchmark/src/network-monitor.ts (1)
NetworkMonitor(6-123)packages/benchmark/src/resource-timing-collector.ts (1)
ResourceTimingCollector(6-169)packages/benchmark/src/perfume-collector.ts (1)
PerfumeCollector(6-142)packages/runner/src/performance-aggregator.ts (1)
PerformanceAggregator(29-372)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(1-13)
packages/runner/src/performance-aggregator.ts (2)
packages/benchmark/src/types.ts (8)
CoreWebVitals(214-225)CookieBannerData(93-101)CookieBannerMetrics(78-91)NetworkRequest(104-111)NetworkMetrics(113-116)ResourceTimingData(126-211)Config(13-56)PerfumeMetrics(228-259)packages/runner/src/types.ts (10)
CoreWebVitals(10-10)CookieBannerData(8-8)CookieBannerMetrics(9-9)NetworkRequest(12-12)NetworkMetrics(11-11)ResourceTimingData(14-14)Config(6-6)PerfumeMetrics(13-13)BenchmarkDetails(24-156)BenchmarkResult(158-271)
packages/cookiebench-cli/src/commands/db.ts (3)
packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/runner/src/utils.ts (1)
ONE_SECOND(5-5)
packages/runner/src/types.ts (2)
packages/cookiebench-cli/src/types/index.ts (3)
ServerInfo(6-6)BenchmarkDetails(3-3)BenchmarkResult(4-4)packages/runner/src/index.ts (3)
ServerInfo(21-21)BenchmarkDetails(10-10)BenchmarkResult(11-11)
packages/benchmark/src/cookie-banner-collector.ts (3)
packages/benchmark/src/types.ts (5)
Config(13-56)CookieBannerMetrics(78-91)WindowWithCookieMetrics(65-76)LayoutShiftEntry(59-62)CookieBannerData(93-101)packages/benchmark/src/bundle-strategy.ts (1)
determineBundleStrategy(4-21)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(1-13)
packages/cookiebench-cli/src/commands/results.ts (6)
packages/benchmark/src/types.ts (1)
Config(13-56)packages/cookiebench-cli/src/types/index.ts (2)
Config(5-5)BenchmarkScores(10-38)packages/runner/src/utils.ts (2)
formatTime(18-23)ONE_SECOND(5-5)packages/cookiebench-cli/src/utils/constants.ts (18)
KILOBYTE(9-9)SCORE_THRESHOLD_POOR(13-13)SCORE_THRESHOLD_FAIR(14-14)CLS_DECIMAL_PLACES(10-10)PERCENTAGE_DIVISOR(6-6)CLS_THRESHOLD_GOOD(17-17)CLS_THRESHOLD_FAIR(18-18)COL_WIDTH_NAME(21-21)COL_WIDTH_CHART_PADDING(27-27)MAX_FILENAME_LENGTH(30-30)TRUNCATED_FILENAME_LENGTH(31-31)MIN_DURATION_THRESHOLD(34-34)COL_WIDTH_TYPE(22-22)COL_WIDTH_SOURCE(23-23)COL_WIDTH_SIZE(24-24)COL_WIDTH_DURATION(25-25)COL_WIDTH_TAGS(26-26)DEFAULT_DOM_SIZE(4-4)packages/cookiebench-cli/src/utils/scoring.ts (1)
calculateScores(1029-1398)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)
packages/cookiebench-cli/src/commands/benchmark.ts (8)
packages/runner/src/types.ts (2)
BenchmarkResult(158-271)ServerInfo(18-21)packages/cookiebench-cli/src/utils/constants.ts (6)
DEFAULT_THIRD_PARTY_DOMAINS(5-5)PERCENTAGE_DIVISOR(6-6)DEFAULT_DOM_SIZE(4-4)HALF_SECOND(2-2)DEFAULT_ITERATIONS(3-3)SEPARATOR_WIDTH(7-7)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/runner/src/benchmark-runner.ts (2)
runSingleBenchmark(37-123)BenchmarkRunner(15-187)packages/runner/src/utils.ts (1)
readConfig(6-16)packages/runner/src/server.ts (2)
buildAndServeNextApp(6-71)cleanupServer(73-77)packages/cookiebench-cli/src/utils/scoring.ts (2)
calculateScores(1029-1398)printScores(1402-1441)packages/cookiebench-cli/src/commands/results.ts (1)
resultsCommand(884-1114)
packages/benchmark/src/perfume-collector.ts (1)
packages/benchmark/src/types.ts (2)
WindowWithPerfumeMetrics(261-277)PerfumeMetrics(228-259)
packages/cookiebench-cli/src/utils/scoring.ts (2)
packages/runner/src/utils.ts (1)
ONE_SECOND(5-5)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)
🪛 LanguageTool
packages/cookiebench-cli/README.md
[uncategorized] ~373-~373: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ice overhead 4. Transparency (15%): Open source status, documentation, licensing 5. **U...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🪛 markdownlint-cli2 (0.18.1)
packages/cookiebench-cli/README.md
99-99: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
99-99: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
170-170: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
170-170: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
336-336: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
🔇 Additional comments (46)
benchmarks/with-enzuzo/next.config.ts (1)
1-1: Stylistic import update aligns with broader refactoring.This quote-style normalization is consistent with the formatting updates across multiple benchmark configs mentioned in the PR. The change has no functional impact.
benchmarks/with-enzuzo/config.json (1)
2-39: Formatting sync looks good.Thanks for normalizing the indentation—keeping these configs tidy helps reduce future diff noise.
benchmarks/with-cookie-control/config.json (1)
2-39: Formatting refresh acknowledged.Indentation is consistent now, which will keep future semantic diffs easy to spot.
packages/cookiebench-cli/rslib.config.ts (1)
1-23: Config aligns with the new bundling story.ESM bundle output with a clean dist matches the rest of the toolchain—looks solid.
benchmarks/with-enzuzo/tsconfig.json (1)
2-10: No issues with the formatting pass.Whitespace changes only; thanks for keeping the project configs consistent.
tsconfig.json (1)
4-8: LGTM! Good type safety improvements.The addition of
strictNullChecks: trueenhances type safety across the project. The multi-line formatting of arrays improves readability and makes diffs clearer for future changes.Also applies to: 20-21, 23-29
packages/benchmark/README.md (1)
1-113: LGTM! Comprehensive package documentation.The README provides clear documentation of the benchmark package with well-structured sections covering installation, usage examples, API reference, and type definitions. The code examples are practical and demonstrate proper usage patterns.
packages/runner/rslib.config.ts (1)
1-14: LGTM! Clean library configuration.The rslib configuration is properly set up for a Node.js library with ESM output, TypeScript declaration generation, and modern ES2021 syntax. This aligns well with the monorepo's restructuring goals.
benchmarks/with-usercentrics/package.json (1)
4-9: LGTM! Consistent with refactoring pattern.The changes align with the CLI refactoring seen across all benchmark packages in this PR. The
benchmarkscript removal and CLI dependency rename from@cookiebench/clitocookiebenchare consistent with the broader restructuring effort.Also applies to: 22-22
turbo.json (1)
5-27: No build task configuration issues identified.The renamed
cookiebenchpackage (previously@cookiebench/cli) contains a build script ("build": "rslib build"), and turbo's dependency pattern"dependsOn": ["^build"]will automatically discover it. The removal of task definitions from turbo.json is correct since they will be auto-discovered. The@cookiebench/wwwpackage has been removed entirely as part of the refactoring.benchmarks/with-c15t-react/package.json (1)
4-9: I need to verify whether the removal of the benchmark script from the individual package breaks benchmark execution for that specific package, and check the broader benchmark workflow structure.I need to check the turbo configuration and verify the cookiebench CLI setup to ensure benchmarks can be executed properly.
Based on my verification, the workflow has been properly restructured:
Root-level benchmark execution infrastructure is in place. Turbo scripts should only be written in the root package.json, as writing turbo commands into individual package.json files can lead to recursively calling turbo. The removal of the
benchmarkscript frombenchmarks/with-c15t-react/package.jsonis correct. Specific packages can be filtered using turbo build --filter, allowing benchmarks to execute through the root-level"benchmark": "pnpm exec cookiebench benchmark"script with the renamedcookiebenchCLI dependency.benchmarks/with-onetrust/package.json (1)
22-22: LGTM!The dependency update from
@cookiebench/clitocookiebenchaligns correctly with the PR's CLI refactoring objectives.package.json (2)
5-12: LGTM!The script updates correctly use
pnpm exec cookiebenchto invoke the renamed CLI, ensuring proper workspace resolution. The command structure is consistent across all three benchmark-related scripts.
17-21: LGTM!The addition of workspace-scoped packages (
@consentio/benchmark,@consentio/runner, andcookiebench) correctly supports the modularized architecture described in the PR objectives.packages/benchmark/rslib.config.ts (1)
1-14: LGTM!The rslib configuration is appropriate for a Node.js library package. ESM format with ES2021 syntax and TypeScript declarations will ensure proper module resolution and type safety for consumers.
benchmarks/with-iubenda/package.json (1)
22-22: LGTM!Dependency migration from
@cookiebench/clitocookiebenchis consistent with the broader refactoring.benchmarks/with-osano/package.json (1)
22-22: LGTM!The dependency update to
cookiebenchaligns with the CLI refactoring objectives.benchmarks/with-c15t-nextjs/package.json (1)
24-24: LGTM!The dependency migration to
cookiebenchis consistent with the broader CLI refactoring across all benchmark packages..cursor/rules/ultracite.mdc (1)
1-227: ****The review comment misunderstands the configuration setup. The
biome.jsoncalready extends Ultracite rules ("extends": ["ultracite/core", "ultracite/next"]), so Biome and Ultracite work together, not in conflict. The.cursor/rules/ultracite.mdcfile is a Cursor IDE editor configuration (alwaysApply: false is an editor-specific setting), separate from the Biome linter. These are complementary tools: Biome handles actual linting/formatting, while Cursor IDE rules provide editor hints. No duplication or consolidation needed.Likely an incorrect or invalid review comment.
packages/cookiebench-cli/base.json (1)
9-9: The CLI base.json file is not actively used by the package and has no practical impact, but the concern about DOM types is valid.The file
packages/cookiebench-cli/base.jsoncontains"DOM"and"DOM.Iterable"in its lib array, but this configuration is not used anywhere in the CLI package. The actual TypeScript configuration used by the CLI istsconfig.json, which does not include a lib array and therefore does not include DOM types. Verification confirms:
- The CLI's actual tsconfig.json (used by
tsc --noEmit) has no "lib" compiler option- The base.json file is not referenced in rslib.config.ts, package.json, or any source code
- There are no DOM API usages in the CLI codebase
- The package has no browser automation dependencies and runs exclusively in Node.js
Your concern about DOM types being inappropriate for a Node.js CLI is technically correct, but since base.json is not being used by the CLI at all, it has zero impact on type checking or runtime behavior. This appears to be unused/dead configuration that should be investigated for removal or clarification of its intended purpose.
benchmarks/with-cookie-control/app/layout.tsx (1)
1-2: LGTM! Formatting changes align with project style.The quote style changes are purely cosmetic and consistent with the broader formatting updates across the PR.
Also applies to: 5-5
benchmarks/with-cookie-control/next.config.ts (1)
1-1: LGTM! Formatting change aligns with project style.The quote style change is purely cosmetic.
benchmarks/with-enzuzo/package.json (1)
22-22: LGTM! Dependency update aligns with CLI refactoring.The change from
@cookiebench/clitocookiebenchis consistent with the broader refactoring across all benchmark packages in this PR.benchmarks/with-cookie-control/tsconfig.json (1)
2-10: LGTM! Formatting changes improve consistency.The indentation and formatting adjustments have no functional impact.
benchmarks/with-enzuzo/app/layout.tsx (1)
1-2: LGTM! Formatting changes align with project style.The quote style and indentation changes are purely cosmetic and consistent with the broader formatting updates across the PR.
Also applies to: 5-5, 18-18
benchmarks/with-cookie-control/package.json (1)
22-22: LGTM! Dependency update aligns with CLI refactoring.The change from
@cookiebench/clitocookiebenchis consistent with the broader refactoring across all benchmark packages in this PR.packages/cookiebench-cli/README.md (1)
1-394: Excellent comprehensive documentation!The README provides thorough coverage of installation, usage, commands, configuration, and development workflows. The structure is clear, examples are helpful, and the coverage of both interactive and direct modes is valuable.
benchmarks/with-cookie-yes/package.json (1)
22-22: LGTM! Dependency migration aligns with CLI refactoring.The addition of the
cookiebenchworkspace dependency is consistent with the PR's objective to rename the CLI package from@cookiebench/clitocookiebench.packages/runner/README.md (1)
1-150: Well-structured package documentation.The README comprehensively documents the runner package with clear examples for basic usage, server management, and remote benchmarking scenarios. The API reference and configuration schema provide good developer guidance.
packages/runner/tsconfig.json (1)
1-15: LGTM! Standard TypeScript configuration.The compiler options are appropriately configured for a modern library package with strict type checking, ES2020 target, and declaration generation enabled.
packages/benchmark/package.json (1)
1-33: LGTM! Well-configured package manifest.The package manifest properly defines ESM exports, includes appropriate build tooling, and declares necessary dependencies for benchmark collection functionality.
packages/cookiebench-cli/src/index.ts (1)
1-135: Well-structured CLI entry point with proper command routing.The implementation correctly:
- Initializes logging with environment-driven log levels
- Gates admin commands with proper access checks
- Supports both direct command invocation and interactive mode
- Handles cancellation and errors gracefully
- Routes to appropriate command handlers
packages/benchmark/src/bundle-strategy.ts (1)
4-21: Duplication resolved – refactoring successfully completed.The old
bundle-strategy.tsfile has been removed, anddetermineBundleStrategyis now exclusively maintained in the@consentio/benchmarkpackage. The function is properly exported via the package's public API (index.ts) and used internally without any orphaned references. The migration is complete.packages/runner/src/types.ts (1)
1-291: LGTM!The type definitions are comprehensive and well-structured. The re-exports from
@consentio/benchmarkpromote code reuse, and theBenchmarkDetailsandBenchmarkResulttypes provide clear contracts for benchmark data.packages/cookiebench-cli/src/utils/constants.ts (1)
1-34: LGTM!The constant definitions are clear and well-organized. The values are appropriate for their use cases across the CLI (timing constants, UI dimensions, thresholds, etc.).
packages/runner/src/performance-aggregator.ts (6)
123-196: Verify the intentional duplication of cookieBanner and thirdParty structures.The method builds
cookieBannerstructures in two places:
- Lines 163:
timing.cookieBanner(with render/interaction timing)- Lines 177-183: Root-level
cookieBanner(with detection metadata)Similarly,
thirdPartyappears in:
- Lines 164-168:
timing.thirdParty- Lines 184-194: Root-level
thirdPartyWhile this may align with the
BenchmarkDetailstype definition, verify that this duplication is intentional and that both structures serve distinct purposes rather than being redundant.
38-49: TTI calculation uses domain-specific heuristic.The TTI calculation uses
Math.maxof FCP, DOM complete timing, and banner interactive time plus a 1000ms buffer. This is a simplified heuristic specific to cookie banner benchmarking rather than the standard TTI metric from Lighthouse/web vitals.Ensure this approach aligns with your benchmarking objectives, as it may not match standard TTI definitions used in broader web performance contexts.
201-212: LGTM!The network impact calculation is straightforward and correctly aggregates size and duration from network requests.
342-372: LGTM!The logging method provides comprehensive debug information with clear bundle strategy determination. Using
logger.debugis appropriate for detailed performance metrics.
75-94: ****The hardcoded values at lines 81-82 and 89-90 are intentional and correct. The DNS and connection timing data collected by
ResourceTimingCollectoris not included in theNetworkRequesttype and therefore never flows tobuildThirdPartyMetrics. The function only receives aggregated impact metrics (totalImpactandtotalDownloadTime), not individual timing breakdowns. Defaulting these fields to 0 is the appropriate design given the available data.Likely an incorrect or invalid review comment.
54-70: Interaction timing values create zero-duration measurement; clarify intended semantics.Lines 61-62 map both
interactionStartandinteractionEndto the same value (bannerInteractiveTime), resulting in zero interaction duration. The source data providesbannerHydrationTime(available inCookieBannerData), which is currently unused and could represent the actual interaction window if that's the intended semantics.Verify whether:
- Zero interaction duration is intentional (measuring only "time to interactive" moment)
interactionEndshould use a different calculation, such asbannerInteractiveTime + bannerHydrationTime, if distinct interaction phases need to be tracked- The semantic meaning of
interactionStartandinteractionEndfields aligns with current implementationpackages/cookiebench-cli/src/utils/scoring.ts (5)
19-42: LGTM! Well-structured type enhancements.The additions to
MetricsDataproperly integrate Perfume.js metrics with appropriate optional markers. ThetimeToFirstByte,interactionToNextPaint, andnetworkInformationfields are correctly typed and align with the performance-aggregator integration.
369-421: LGTM! TTFB and INP scoring logic is well-implemented.The new TTFB and INP scoring additions enhance the performance assessment with modern web vitals. The thresholds align with industry best practices, and the neutral score (5 points) for INP when no interactions are detected is a reasonable fallback.
879-909: LGTM! Enhanced insights provide valuable context.The new insights for network quality, TTFB, and INP enrich the scoring output with actionable feedback. The network information context (lines 879-885) is particularly useful for understanding test conditions, and the TTFB/INP thresholds align with the scoring logic.
946-956: LGTM! Actionable recommendations for TTFB and INP.The new recommendations provide concrete guidance for improving TTFB (server optimization, CDN, caching) and INP (JavaScript execution, event handler performance). The thresholds (>200ms) align with the "Good" threshold from the scoring logic.
1029-1071: LGTM! Backward-compatible API extension.The
calculateScoresfunction signature properly extends to accept Perfume.js metrics (timeToFirstByte,interactionToNextPaint) and network information. The new parameters are optional, maintaining backward compatibility, and are correctly integrated into the metrics data structure.Also applies to: 1205-1208
| @@ -1,5 +1,6 @@ | |||
| /// <reference types="next" /> | |||
| /// <reference types="next/image-types/global" /> | |||
| import "./.next/types/routes.d.ts"; | |||
There was a problem hiding this comment.
🧩 Analysis chain
Verify manual modification of auto-generated file.
This file explicitly states it should not be edited (line 5-6), yet a manual import has been added. Additionally, the import references .next/types/routes.d.ts, which is a build-time artifact that may not exist before the first build, potentially causing TypeScript errors.
Next.js typed routes are typically configured via next.config.js using experimental.typedRoutes: true (or typedRoutes: true in Next.js 15+), and the types are automatically included without manual imports.
Please confirm whether this manual import is necessary or if Next.js configuration should be used instead.
🏁 Script executed:
#!/bin/bash
# Description: Check if typed routes are properly configured in next.config.js
# and verify if this manual import is necessary
# Check for typed routes configuration in Next.js config files
echo "=== Checking Next.js config for typed routes configuration ==="
fd -e js -e ts -e mjs 'next.config' benchmarks/with-c15t-nextjs/ --exec cat {}
echo -e "\n=== Checking if .next/types/routes.d.ts exists (build artifact) ==="
if [ -f "benchmarks/with-c15t-nextjs/.next/types/routes.d.ts" ]; then
echo "File exists"
else
echo "File does not exist (expected - it's a build artifact)"
fiLength of output: 318
---
🌐 Web query:
```web
Next.js typed routes import next-env.d.ts manual configuration
💡 Result:
Short answer — don’t edit Next’s generated next-env.d.ts; instead make sure it’s included and add your own declarations. Steps & minimal configs:
-
Ensure Next’s generated file is included in tsconfig.json (Next will regenerate it on dev/build). Example include array:
{
"include": [
"next-env.d.ts",
".next/types//*.ts", // required when using typedRoutes / generated types
"/.ts",
"**/.tsx"
],
"exclude": ["node_modules"]
}
(Next docs recommend exactly this pattern). [1][2] -
If you need custom global declarations, create a separate d.ts (e.g. src/types.d.ts or global.d.ts) — do NOT modify next-env.d.ts (it’s overwritten). Add that filename to tsconfig.json include. Example:
- src/global.d.ts
declare namespace MyApp { /* ... */ } - tsconfig.json include: ["src/global.d.ts", "next-env.d.ts", ".next/types//*.ts", "/.ts", "**/.tsx"]. [2]
- src/global.d.ts
-
If you need to “manually” provide the next types (e.g. you didn’t use create-next-app), you can add a next-env.d.ts containing the standard reference and keep it under version control:
///
///
But be aware Next will regenerate/overwrite this file during dev/build — prefer creating your own d.ts for custom types. [2] -
For typed routes (typedRoutes: true in next.config), include the generated .next/types/**/*.ts so TypeScript picks up .next’s route/type definitions. Run next dev / next build (or next typegen) to generate those types. [1]
Sources:
- Next.js TypeScript docs (includes guidance about next-env.d.ts, including .next types, and creating separate declaration files). [1][2]
---
🏁 Script executed:
```shell
#!/bin/bash
# Check tsconfig.json and verify the full context
echo "=== Checking tsconfig.json in benchmarks/with-c15t-nextjs/ ==="
cat benchmarks/with-c15t-nextjs/tsconfig.json
echo -e "\n=== Checking the full content of next-env.d.ts ==="
cat benchmarks/with-c15t-nextjs/next-env.d.ts
Length of output: 614
Remove manual import from next-env.d.ts; configure typed routes in next.config.js instead.
The file explicitly states it should not be edited. The manual import on line 3 (import "./.next/types/routes.d.ts") violates this guidance and references a build artifact that doesn't exist pre-build, causing TypeScript compilation errors before the first build.
If you need typed routes, configure typedRoutes: true in next.config.js and let Next.js auto-generate the types. The tsconfig.json already includes the correct path (.next/types/**/*.ts), so no manual import is needed.
Action:
- Remove the import statement from line 3
- If typed routes are needed for this benchmark, add
typedRoutes: trueto next.config.js and runnext buildornext typegen
🤖 Prompt for AI Agents
In benchmarks/with-c15t-nextjs/next-env.d.ts around line 3, remove the manual
import `import "./.next/types/routes.d.ts"` because it references a build
artifact and causes TS errors pre-build; instead, delete that line and if typed
routes are required enable `typedRoutes: true` in next.config.js and run `next
build` or `next typegen` so Next.js auto-generates the `.next/types` files.
| "target": "ES2020", | ||
| "module": "ESNext", | ||
| "moduleResolution": "bundler", | ||
| "esModuleInterop": true, | ||
| "strict": true, | ||
| "skipLibCheck": true, | ||
| "declaration": true, | ||
| "outDir": "dist", | ||
| "rootDir": "src" |
There was a problem hiding this comment.
Reconsider moduleResolution: "bundler" for this package.
bundler is great for app builds, but for a library emitting declarations it can leak extensionless specifiers into .d.ts files. Those specifiers break consumers compiling with Node-compatible resolution (node16/nodenext) because TypeScript won’t rewrite them to .js, and Node rejects the extensionless import at runtime. The TypeScript docs explicitly recommend sticking to the Node modes (or bundling the declaration output) when you plan to publish library code.(runebook.dev)
A straightforward fix is to switch to the Node-aligned setting:
- "moduleResolution": "bundler",
+ "moduleResolution": "nodenext",If you truly need bundler, please ensure the declarations are post-processed so that every relative specifier has the correct .js (or .mjs) suffix before publishing.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| "target": "ES2020", | |
| "module": "ESNext", | |
| "moduleResolution": "bundler", | |
| "esModuleInterop": true, | |
| "strict": true, | |
| "skipLibCheck": true, | |
| "declaration": true, | |
| "outDir": "dist", | |
| "rootDir": "src" | |
| "target": "ES2020", | |
| "module": "ESNext", | |
| "moduleResolution": "nodenext", | |
| "esModuleInterop": true, | |
| "strict": true, | |
| "skipLibCheck": true, | |
| "declaration": true, | |
| "outDir": "dist", | |
| "rootDir": "src" |
🤖 Prompt for AI Agents
In packages/benchmark/tsconfig.json around lines 3 to 11, moduleResolution is
set to "bundler", which can emit extensionless specifiers into .d.ts files that
break consumers using Node-compatible resolution; change moduleResolution to a
Node-aligned mode such as "node16" or "nodenext" (and adjust related settings
like module if needed) so TypeScript rewrites imports correctly for published
declarations, or if you must keep "bundler" add a post-processing step that
rewrites all relative specifiers in emitted .d.ts files to include the
appropriate .js/.mjs extensions before publishing.
| - ✅ Summary report at the end | ||
|
|
||
| **Example:** | ||
| ``` |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Minor markdown formatting issues.
Static analysis tools flagged some markdown formatting concerns:
- Fenced code blocks at lines 99, 170, and 336 should be surrounded by blank lines and have language specifiers
- Line 373: "Open source" could be hyphenated as "open-source" when used as a compound adjective
These are minor style issues that don't affect functionality.
Also applies to: 170-170, 336-336
🧰 Tools
🪛 markdownlint-cli2 (0.18.1)
99-99: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
99-99: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
In packages/cookiebench-cli/README.md around lines 99, 170, 336 and line 373:
the fenced code blocks at lines 99, 170, and 336 need blank lines before and
after the triple-backticks and should include a language specifier (e.g.,
```bash or ```js) to satisfy markdown linters; edit those blocks to add a blank
line above and below each fence and append the appropriate language token after
the opening backticks. Also update line 373 by changing the phrase "Open source"
to the hyphenated compound adjective "open-source".
| { | ||
| "compilerOptions": { | ||
| "target": "ES2020", | ||
| "module": "ESNext", | ||
| "moduleResolution": "bundler", | ||
| "esModuleInterop": true, | ||
| "strict": true, | ||
| "skipLibCheck": true, | ||
| "outDir": "dist", | ||
| "rootDir": "src" | ||
| }, | ||
| "include": ["src/**/*"], | ||
| "exclude": ["node_modules", "dist"] | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider adding declaration file generation.
While this CLI package configuration is functional, it lacks "declaration": true for generating TypeScript declaration files. Although CLIs typically don't require declarations, adding this option would improve consistency with other packages in the monorepo (e.g., packages/runner/rslib.config.ts has dts: true) and enable better IDE support if other packages import utilities from the CLI.
Apply this diff if declaration files are desired:
"strict": true,
"skipLibCheck": true,
+ "declaration": true,
"outDir": "dist",
"rootDir": "src"📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| { | |
| "compilerOptions": { | |
| "target": "ES2020", | |
| "module": "ESNext", | |
| "moduleResolution": "bundler", | |
| "esModuleInterop": true, | |
| "strict": true, | |
| "skipLibCheck": true, | |
| "outDir": "dist", | |
| "rootDir": "src" | |
| }, | |
| "include": ["src/**/*"], | |
| "exclude": ["node_modules", "dist"] | |
| } | |
| { | |
| "compilerOptions": { | |
| "target": "ES2020", | |
| "module": "ESNext", | |
| "moduleResolution": "bundler", | |
| "esModuleInterop": true, | |
| "strict": true, | |
| "skipLibCheck": true, | |
| "declaration": true, | |
| "outDir": "dist", | |
| "rootDir": "src" | |
| }, | |
| "include": ["src/**/*"], | |
| "exclude": ["node_modules", "dist"] | |
| } |
🤖 Prompt for AI Agents
packages/cookiebench-cli/tsconfig.json lines 1-14: the TS config doesn’t enable
declaration file generation; to align with monorepo conventions and improve
IDE/consumer support, set compilerOptions.declaration to true (and optionally
add declarationDir, e.g., "dist/types" or similar) so .d.ts files are emitted to
your build output; ensure declaration generation does not conflict with existing
outDir/rootDir settings.
| // Setup monitoring and detection | ||
| await this.networkMonitor.setupMonitoring(page); | ||
| await this.cookieBannerCollector.setupDetection(page); | ||
| await this.perfumeCollector.setupPerfume(page); | ||
|
|
||
| // Navigate to the page | ||
| this.logger.debug(`Navigating to: ${url}`); | ||
| await page.goto(url, { waitUntil: "networkidle" }); | ||
|
|
||
| // Wait for the specified element | ||
| await this.waitForElement(page); | ||
|
|
||
| // Wait for network to be idle | ||
| this.logger.debug("Waiting for network idle..."); | ||
| await page.waitForLoadState("networkidle"); | ||
|
|
||
| // Collect core web vitals from playwright-performance-metrics (primary source) | ||
| this.logger.debug("Collecting core web vitals..."); | ||
| const coreWebVitals = await collector.collectMetrics(page, { | ||
| timeout: BENCHMARK_CONSTANTS.METRICS_TIMEOUT, | ||
| retryTimeout: BENCHMARK_CONSTANTS.METRICS_RETRY_TIMEOUT, | ||
| }); | ||
|
|
||
| this.logger.debug("Core web vitals collected:", { | ||
| fcp: coreWebVitals.paint?.firstContentfulPaint, | ||
| lcp: coreWebVitals.largestContentfulPaint, | ||
| cls: coreWebVitals.cumulativeLayoutShift, | ||
| tbt: coreWebVitals.totalBlockingTime, | ||
| }); | ||
|
|
||
| // Collect Perfume.js metrics (supplementary - TTFB, navigation timing, network info) | ||
| this.logger.debug("Collecting Perfume.js supplementary metrics..."); | ||
| const perfumeMetrics = await this.perfumeCollector.collectMetrics(page); | ||
| this.logger.debug("Perfume.js metrics:", perfumeMetrics); | ||
|
|
||
| // Collect cookie banner specific metrics | ||
| const cookieBannerData = | ||
| await this.cookieBannerCollector.collectMetrics(page); | ||
| this.logger.debug("Cookie banner metrics:", cookieBannerData); | ||
|
|
||
| // Collect detailed resource timing data | ||
| const resourceMetrics = await this.resourceTimingCollector.collect(page); | ||
|
|
||
| // Get network metrics | ||
| const networkRequests = this.networkMonitor.getNetworkRequests(); | ||
| const networkMetrics = this.networkMonitor.getMetrics(); | ||
|
|
||
| // Aggregate all metrics | ||
| const finalMetrics = this.performanceAggregator.aggregateMetrics({ | ||
| coreWebVitals, | ||
| cookieBannerData, | ||
| cookieBannerMetrics, | ||
| networkRequests, | ||
| networkMetrics, | ||
| resourceMetrics, | ||
| config: this.config, | ||
| perfumeMetrics, | ||
| }); | ||
|
|
There was a problem hiding this comment.
Third-party/network metrics never populate
NetworkMonitor’s third-party detection currently runs !url.includes(new URL(url).hostname), which is always false; it also leaves duration at 0. As a result, networkMonitor.getMetrics() and networkMonitor.getNetworkRequests() feed only zeroes into aggregateMetrics, so every run reports zero third-party impact and zero download time. That breaks the headline scoring story of this refactor.
Please pass the page’s hostname into setupMonitoring and update NetworkMonitor to compare new URL(request.url()).hostname against that host and to record timing (e.g., via response.timing()). For example, this file should provide the host:
- await this.networkMonitor.setupMonitoring(page);
+ const targetHost = new URL(url).hostname;
+ await this.networkMonitor.setupMonitoring(page, targetHost);and NetworkMonitor should use that hostname when classifying requests and populate duration. Without these fixes, the CLI reports misleading metrics.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // Setup monitoring and detection | |
| await this.networkMonitor.setupMonitoring(page); | |
| await this.cookieBannerCollector.setupDetection(page); | |
| await this.perfumeCollector.setupPerfume(page); | |
| // Navigate to the page | |
| this.logger.debug(`Navigating to: ${url}`); | |
| await page.goto(url, { waitUntil: "networkidle" }); | |
| // Wait for the specified element | |
| await this.waitForElement(page); | |
| // Wait for network to be idle | |
| this.logger.debug("Waiting for network idle..."); | |
| await page.waitForLoadState("networkidle"); | |
| // Collect core web vitals from playwright-performance-metrics (primary source) | |
| this.logger.debug("Collecting core web vitals..."); | |
| const coreWebVitals = await collector.collectMetrics(page, { | |
| timeout: BENCHMARK_CONSTANTS.METRICS_TIMEOUT, | |
| retryTimeout: BENCHMARK_CONSTANTS.METRICS_RETRY_TIMEOUT, | |
| }); | |
| this.logger.debug("Core web vitals collected:", { | |
| fcp: coreWebVitals.paint?.firstContentfulPaint, | |
| lcp: coreWebVitals.largestContentfulPaint, | |
| cls: coreWebVitals.cumulativeLayoutShift, | |
| tbt: coreWebVitals.totalBlockingTime, | |
| }); | |
| // Collect Perfume.js metrics (supplementary - TTFB, navigation timing, network info) | |
| this.logger.debug("Collecting Perfume.js supplementary metrics..."); | |
| const perfumeMetrics = await this.perfumeCollector.collectMetrics(page); | |
| this.logger.debug("Perfume.js metrics:", perfumeMetrics); | |
| // Collect cookie banner specific metrics | |
| const cookieBannerData = | |
| await this.cookieBannerCollector.collectMetrics(page); | |
| this.logger.debug("Cookie banner metrics:", cookieBannerData); | |
| // Collect detailed resource timing data | |
| const resourceMetrics = await this.resourceTimingCollector.collect(page); | |
| // Get network metrics | |
| const networkRequests = this.networkMonitor.getNetworkRequests(); | |
| const networkMetrics = this.networkMonitor.getMetrics(); | |
| // Aggregate all metrics | |
| const finalMetrics = this.performanceAggregator.aggregateMetrics({ | |
| coreWebVitals, | |
| cookieBannerData, | |
| cookieBannerMetrics, | |
| networkRequests, | |
| networkMetrics, | |
| resourceMetrics, | |
| config: this.config, | |
| perfumeMetrics, | |
| }); | |
| // Setup monitoring and detection | |
| const targetHost = new URL(url).hostname; | |
| await this.networkMonitor.setupMonitoring(page, targetHost); | |
| await this.cookieBannerCollector.setupDetection(page); | |
| await this.perfumeCollector.setupPerfume(page); | |
| // Navigate to the page | |
| this.logger.debug(`Navigating to: ${url}`); | |
| await page.goto(url, { waitUntil: "networkidle" }); | |
| // Wait for the specified element | |
| await this.waitForElement(page); | |
| // Wait for network to be idle | |
| this.logger.debug("Waiting for network idle..."); | |
| await page.waitForLoadState("networkidle"); | |
| // Collect core web vitals from playwright-performance-metrics (primary source) | |
| this.logger.debug("Collecting core web vitals..."); | |
| const coreWebVitals = await collector.collectMetrics(page, { | |
| timeout: BENCHMARK_CONSTANTS.METRICS_TIMEOUT, | |
| retryTimeout: BENCHMARK_CONSTANTS.METRICS_RETRY_TIMEOUT, | |
| }); | |
| this.logger.debug("Core web vitals collected:", { | |
| fcp: coreWebVitals.paint?.firstContentfulPaint, | |
| lcp: coreWebVitals.largestContentfulPaint, | |
| cls: coreWebVitals.cumulativeLayoutShift, | |
| tbt: coreWebVitals.totalBlockingTime, | |
| }); | |
| // Collect Perfume.js metrics (supplementary - TTFB, navigation timing, network info) | |
| this.logger.debug("Collecting Perfume.js supplementary metrics..."); | |
| const perfumeMetrics = await this.perfumeCollector.collectMetrics(page); | |
| this.logger.debug("Perfume.js metrics:", perfumeMetrics); | |
| // Collect cookie banner specific metrics | |
| const cookieBannerData = | |
| await this.cookieBannerCollector.collectMetrics(page); | |
| this.logger.debug("Cookie banner metrics:", cookieBannerData); | |
| // Collect detailed resource timing data | |
| const resourceMetrics = await this.resourceTimingCollector.collect(page); | |
| // Get network metrics | |
| const networkRequests = this.networkMonitor.getNetworkRequests(); | |
| const networkMetrics = this.networkMonitor.getMetrics(); | |
| // Aggregate all metrics | |
| const finalMetrics = this.performanceAggregator.aggregateMetrics({ | |
| coreWebVitals, | |
| cookieBannerData, | |
| cookieBannerMetrics, | |
| networkRequests, | |
| networkMetrics, | |
| resourceMetrics, | |
| config: this.config, | |
| perfumeMetrics, | |
| }); |
🤖 Prompt for AI Agents
In packages/runner/src/benchmark-runner.ts around lines 52 to 110, the
NetworkMonitor is currently set up without the page hostname so its third-party
detection `!url.includes(new URL(url).hostname)` always fails and durations stay
zero; update the call to this.networkMonitor.setupMonitoring(page) to pass the
page's hostname (e.g., new URL(url).hostname or await page.url() hostname) so
NetworkMonitor has the origin to compare against, and then update NetworkMonitor
to classify requests by comparing new URL(request.url()).hostname !== host and
to compute/record duration from the response timing (use response.timing() or
available timing fields to set a numeric duration), ensuring getMetrics() and
getNetworkRequests() return proper third-party flags and non-zero durations for
aggregation.
…le benchmarks. Corrected typos in README, standardized bundle type naming from 'iffe' to 'iife', and added Node.js engine requirements in package.json files. Enhanced constants and utility functions by importing shared values for better maintainability.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 19
♻️ Duplicate comments (23)
benchmarks/with-usercentrics/config.json (1)
20-20: Correct bundleType typo.Consistent with bundle type corrections across all benchmark configurations.
benchmarks/with-iubenda/config.json (1)
25-25: Fixed bundleType typo.The typo correction from "iffe" to "iife" addresses the issue previously flagged in the review.
benchmarks/with-cookie-control/config.json (1)
20-20: Correct bundleType value.The bundleType has been corrected to "iife" as part of the systematic bundle type standardization across all benchmark configurations.
benchmarks/with-enzuzo/config.json (1)
20-20: Correct bundleType typo.Aligns with bundle type standardization across benchmark configurations.
benchmarks/with-cookie-yes/config.json (1)
24-24: Correct bundleType value.Consistent with bundle type corrections applied across all benchmark configurations.
packages/cookiebench-cli/src/commands/save.ts (2)
467-472: Critical issue: Third-party request count is incomplete.The
thirdPartyRequestsmetric only counts script resources, excluding third-party CSS, images, fonts, and other resource types. This results in significantly understated third-party request counts being persisted to the API.Apply this diff to count all third-party resources:
thirdPartyRequests: appResults.reduce( (a, b) => - a + b.resources.scripts.filter((s) => s.isThirdParty).length, + a + + b.resources.scripts.filter((r) => r.isThirdParty).length + + b.resources.styles.filter((r) => r.isThirdParty).length + + b.resources.images.filter((r) => r.isThirdParty).length + + b.resources.fonts.filter((r) => r.isThirdParty).length + + b.resources.other.filter((r) => r.isThirdParty).length, 0 ) / appResults.length,
544-550: Critical issue: Multiple metrics are zeroed or incomplete.Several averaged metrics are hardcoded to 0 or only count scripts, shipping inaccurate data to the API:
scriptLoadTimeis hardcoded to 0 instead of averaging actual script load durationsscriptSizeis hardcoded to 0 instead of averaging actual script sizesresourceCountonly counts scripts, excluding styles, images, fonts, and other resourcesApply this diff to populate these metrics correctly:
- scriptLoadTime: 0, + scriptLoadTime: + appResults.reduce( + (total, result) => + total + + result.resources.scripts.reduce( + (sum, script) => sum + script.duration, + 0 + ), + 0 + ) / appResults.length, totalSize: appResults.reduce((a, b) => a + b.size.total, 0) / appResults.length, - scriptSize: 0, + scriptSize: + appResults.reduce( + (total, result) => total + result.size.scripts.total, + 0 + ) / appResults.length, resourceCount: - appResults.reduce((a, b) => a + b.resources.scripts.length, 0) / - appResults.length, + appResults.reduce( + (total, result) => + total + + result.resources.scripts.length + + result.resources.styles.length + + result.resources.images.length + + result.resources.fonts.length + + result.resources.other.length, + 0 + ) / appResults.length,benchmarks/with-enzuzo/package.json (1)
6-6: Port conflict already flagged.The port conflict on 3001 with
benchmarks/with-iubendawas already identified in previous review comments.benchmarks/with-didomi/package.json (1)
2-2: Package name fix already flagged.The correction from "with-dodomi" to "with-didomi" was already identified in previous review comments.
benchmarks/with-osano/package.json (1)
6-6: Port conflict already flagged.The port conflict on 3006 with
benchmarks/with-onetrustwas already identified in previous review comments.benchmarks/with-iubenda/package.json (1)
6-6: Port 3001 conflict already flagged.The dev server port conflict on port 3001 was already identified in a previous review.
packages/shared/src/utils/package-manager.ts (2)
5-37: Refactor nested try-catch and remove code duplication.The function has deeply nested try-catch blocks (3 levels) and duplicates
execSyncimports. This pattern was already flagged in a previous review.Apply this refactor to flatten the logic and improve readability:
export async function getPackageManager(): Promise<{ command: string; args: string[]; }> { + const { execSync } = await import("node:child_process"); + const managers = [ + { name: "npm", command: "npm", args: ["run"] }, + { name: "yarn", command: "yarn", args: [] }, + { name: "pnpm", command: "pnpm", args: [] }, + ]; + + for (const manager of managers) { + try { + execSync(`${manager.name} -v`, { encoding: "utf-8" }); + return { command: manager.command, args: manager.args }; + } catch { + // Try next manager + } + } + + // Default to npm if no package manager is found + return { command: "npm", args: ["run"] }; - try { - const { execSync } = await import("node:child_process"); - const output = execSync("npm -v", { encoding: "utf-8" }); - if (output) { - return { command: "npm", args: ["run"] }; - } - } catch { - try { - const { execSync } = await import("node:child_process"); - const output = execSync("yarn -v", { encoding: "utf-8" }); - if (output) { - return { command: "yarn", args: [] }; - } - } catch { - try { - const { execSync } = await import("node:child_process"); - const output = execSync("pnpm -v", { encoding: "utf-8" }); - if (output) { - return { command: "pnpm", args: [] }; - } - } catch { - // Default to npm if no package manager is found - return { command: "npm", args: ["run"] }; - } - } - } - // Default to npm if no package manager is found - return { command: "npm", args: ["run"] }; }
35-36: Remove unreachable code.Line 36 is unreachable because line 31 always returns in the final catch block. This was already flagged in a previous review.
} catch { // Default to npm if no package manager is found return { command: "npm", args: ["run"] }; } } } - // Default to npm if no package manager is found - return { command: "npm", args: ["run"] }; }packages/cookiebench-cli/src/index.ts (1)
114-128: Consider adding default case for defensive programming.As noted in previous reviews, adding a default case to this switch statement would be defensive programming and eliminate the need for the biome-ignore directive. While the select prompt constrains valid commands, an explicit default provides clarity and guards against future refactoring issues.
Apply this diff:
// biome-ignore lint/style/useDefaultSwitchClause: this is a CLI tool switch (selectedCommand) { case "benchmark": await benchmarkCommand(logger); break; case "results": await resultsCommand(logger); break; case "save": await saveCommand(logger); break; case "db": await dbCommand(logger); break; + default: + logger.error(`Unexpected command: ${selectedCommand}`); + process.exit(1); }packages/cookiebench-cli/package.json (3)
9-11: Shebang configuration still needed for CLI executable.As noted in previous reviews, the bin entry requires a shebang in the built output. Configure rslib to inject it via BannerPlugin in
rslib.config.ts, or add a postbuild script to ensure the file is executable.
22-22: Replace alpha dependency with stable release.As noted in previous reviews,
@clack/prompts@1.0.0-alpha.0should be replaced with the latest stable version0.11.0to avoid potential breaking changes and stability issues.Apply this diff:
- "@clack/prompts": "^1.0.0-alpha.0", + "@clack/prompts": "^0.11.0",
26-26: Move@types/figletto devDependencies.As noted in previous reviews, TypeScript type definitions should be in
devDependenciesto avoid bloating runtime installs.Apply this diff:
"dependencies": { "@c15t/logger": "^1.0.0", "@clack/prompts": "^1.0.0-alpha.0", "@consentio/benchmark": "workspace:*", "@consentio/runner": "workspace:*", "@consentio/shared": "workspace:*", - "@types/figlet": "^1.7.0", "cli-table3": "^0.6.3", "dotenv": "^17.2.3", "figlet": "^1.9.3", "picocolors": "^1.0.0", "pretty-ms": "^9.3.0" }, "devDependencies": { "@rsdoctor/rspack-plugin": "^1.3.6", "@rslib/core": "^0.16.1", "@types/node": "^24.9.2", + "@types/figlet": "^1.7.0", "typescript": "^5.9.3" }packages/runner/src/server.ts (1)
55-71: Terminate the Next.js process when startup fails.If the health check never returns 200, the code throws an error but leaves the spawned server alive, leaking a process and port. Kill the process before surfacing the error.
Apply this diff to fix:
while (retries < maxRetries) { try { const response = await fetch(url); if (response.ok) { logger.success("Server is ready!"); return { serverProcess, url }; } } catch { // Ignore error and retry } await new Promise((resolve) => setTimeout(resolve, ONE_SECOND)); retries += 1; } + if (!serverProcess.killed) { + serverProcess.kill(); + } throw new Error("Server failed to start");packages/cookiebench-cli/src/commands/scores.ts (1)
208-226: Guard against empty result sets before averaging.If a results file exists but
resultsis empty (e.g., a failed/aborted run wrote the file), every average below divides by0, producingNaNscores. Bail out early with a clear error before callingcalculateScores.Apply this diff:
const appResults = result.results; + if (!Array.isArray(appResults) || appResults.length === 0) { + logger.error(`No iterations recorded for ${appName}; cannot compute scores.`); + return; + } const config = await loadConfigForApp(logger, appName);packages/cookiebench-cli/src/commands/db.ts (1)
30-44: Make drizzle-kit invocation path-safe.Building the shell command with
cd ${DB_PACKAGE_PATH}breaks whenever the repo path has spaces (e.g.,C:\Users\Jane Doe\...). Usingcwdkeeps the invocation portable across Windows and macOS.Apply this diff:
function runDrizzleCommand(logger: CliLogger, command: string): void { try { logger.step(`Running: ${color.cyan(`drizzle-kit ${command}`)}`); - execSync(`cd ${DB_PACKAGE_PATH} && pnpm drizzle-kit ${command}`, { + execSync(`pnpm drizzle-kit ${command}`, { stdio: "inherit", encoding: "utf-8", + cwd: DB_PACKAGE_PATH, }); } catch (error) {packages/runner/src/performance-aggregator.ts (2)
104-108: Redundant fallback in division operation.Line 106 uses
(totalBlockingTime || 1)as the denominator, but this fallback is unnecessary since the ternary on line 105 already ensures the division only occurs whentotalBlockingTime > 0.Apply this diff to simplify:
const percentageFromCookies = totalBlockingTime > 0 - ? (cookieBannerEstimate / (totalBlockingTime || 1)) * + ? (cookieBannerEstimate / totalBlockingTime) * PERCENTAGE_MULTIPLIER : 0;
214-334: Consider distinguishing "not measured" from zero.Several metrics default to 0 when not calculated (e.g.,
speedIndexat line 238,domSizeat line 259, and various third-party metrics at lines 289-321). This conflates "not measured" with "measured as zero." Consider usingnullor marking unavailable metrics explicitly to distinguish these cases.packages/cookiebench-cli/src/commands/results.ts (1)
1042-1047: Fix third-party request counting.
thirdPartyRequestsonly counts script resources, so any third-party styles, images, fonts, or "other" assets are ignored. This underreports the network impact category and leads to incorrect scores for apps that load third-party resources outside of JavaScript.Apply this diff:
thirdPartyRequests: appResults.reduce( (a, b) => - a + b.resources.scripts.filter((s) => s.isThirdParty).length, + a + + b.resources.scripts.filter((r) => r.isThirdParty).length + + b.resources.styles.filter((r) => r.isThirdParty).length + + b.resources.images.filter((r) => r.isThirdParty).length + + b.resources.fonts.filter((r) => r.isThirdParty).length + + b.resources.other.filter((r) => r.isThirdParty).length, 0 ) / appResults.length,
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (53)
.gitignore(1 hunks)README.md(2 hunks)benchmarks/baseline/package.json(2 hunks)benchmarks/with-c15t-nextjs/package.json(1 hunks)benchmarks/with-c15t-react/package.json(1 hunks)benchmarks/with-cookie-control/app/layout.tsx(3 hunks)benchmarks/with-cookie-control/config.json(1 hunks)benchmarks/with-cookie-control/package.json(1 hunks)benchmarks/with-cookie-yes/config.json(1 hunks)benchmarks/with-cookie-yes/package.json(1 hunks)benchmarks/with-didomi/app/layout.tsx(3 hunks)benchmarks/with-didomi/package.json(2 hunks)benchmarks/with-enzuzo/config.json(1 hunks)benchmarks/with-enzuzo/package.json(1 hunks)benchmarks/with-iubenda/app/layout.tsx(2 hunks)benchmarks/with-iubenda/config.json(1 hunks)benchmarks/with-iubenda/package.json(1 hunks)benchmarks/with-ketch/config.json(1 hunks)benchmarks/with-ketch/package.json(1 hunks)benchmarks/with-onetrust/app/layout.tsx(2 hunks)benchmarks/with-onetrust/config.json(1 hunks)benchmarks/with-onetrust/package.json(1 hunks)benchmarks/with-osano/config.json(1 hunks)benchmarks/with-osano/package.json(1 hunks)benchmarks/with-usercentrics/config.json(1 hunks)benchmarks/with-usercentrics/package.json(1 hunks)packages/benchmark-schema/schema.json(7 hunks)packages/benchmark/package.json(1 hunks)packages/benchmark/src/constants.ts(1 hunks)packages/cookiebench-cli/package.json(1 hunks)packages/cookiebench-cli/src/commands/benchmark.ts(1 hunks)packages/cookiebench-cli/src/commands/db.ts(1 hunks)packages/cookiebench-cli/src/commands/results.ts(1 hunks)packages/cookiebench-cli/src/commands/save.ts(1 hunks)packages/cookiebench-cli/src/commands/scores.ts(1 hunks)packages/cookiebench-cli/src/index.ts(1 hunks)packages/cookiebench-cli/src/utils/constants.ts(1 hunks)packages/cookiebench-cli/src/utils/index.ts(1 hunks)packages/cookiebench-cli/src/utils/scoring.ts(26 hunks)packages/runner/package.json(1 hunks)packages/runner/src/index.ts(1 hunks)packages/runner/src/performance-aggregator.ts(1 hunks)packages/runner/src/server.ts(1 hunks)packages/shared/README.md(1 hunks)packages/shared/package.json(1 hunks)packages/shared/rslib.config.ts(1 hunks)packages/shared/src/constants.ts(1 hunks)packages/shared/src/index.ts(1 hunks)packages/shared/src/utils/config.ts(1 hunks)packages/shared/src/utils/conversion.ts(1 hunks)packages/shared/src/utils/package-manager.ts(1 hunks)packages/shared/src/utils/time.ts(1 hunks)packages/shared/tsconfig.json(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (17)
packages/shared/src/utils/time.ts (2)
packages/shared/src/index.ts (2)
formatTime(22-22)ONE_SECOND(8-8)packages/shared/src/constants.ts (1)
ONE_SECOND(2-2)
packages/shared/src/utils/package-manager.ts (1)
packages/shared/src/index.ts (1)
getPackageManager(21-21)
packages/shared/src/utils/config.ts (2)
packages/shared/src/index.ts (2)
BaseConfig(13-13)readConfig(13-13)packages/cookiebench-cli/src/utils/index.ts (1)
readConfig(34-36)
packages/cookiebench-cli/src/commands/db.ts (3)
packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/shared/src/constants.ts (1)
ONE_SECOND(2-2)
packages/cookiebench-cli/src/commands/benchmark.ts (7)
packages/runner/src/types.ts (2)
BenchmarkResult(158-271)ServerInfo(18-21)packages/cookiebench-cli/src/utils/constants.ts (4)
DEFAULT_THIRD_PARTY_DOMAINS(4-4)DEFAULT_DOM_SIZE(3-3)DEFAULT_ITERATIONS(2-2)SEPARATOR_WIDTH(5-5)packages/shared/src/constants.ts (2)
PERCENTAGE_DIVISOR(11-11)HALF_SECOND(3-3)packages/shared/src/utils/config.ts (1)
readConfig(14-26)packages/runner/src/server.ts (2)
buildAndServeNextApp(6-71)cleanupServer(73-77)packages/cookiebench-cli/src/utils/scoring.ts (2)
calculateScores(1010-1379)printScores(1383-1422)packages/cookiebench-cli/src/commands/results.ts (1)
resultsCommand(882-1112)
packages/benchmark/src/constants.ts (2)
packages/benchmark/src/index.ts (2)
BENCHMARK_CONSTANTS(5-5)BUNDLE_TYPES(5-5)packages/shared/src/constants.ts (2)
ONE_SECOND(2-2)TTI_BUFFER_MS(14-14)
packages/shared/src/constants.ts (2)
packages/cookiebench-cli/src/utils/index.ts (5)
ONE_SECOND(22-22)HALF_SECOND(20-20)KILOBYTE(21-21)PERCENTAGE_MULTIPLIER(24-24)PERCENTAGE_DIVISOR(23-23)packages/shared/src/index.ts (7)
ONE_SECOND(8-8)HALF_SECOND(6-6)BYTES_TO_KB(5-5)KILOBYTE(7-7)PERCENTAGE_MULTIPLIER(10-10)PERCENTAGE_DIVISOR(9-9)TTI_BUFFER_MS(11-11)
packages/cookiebench-cli/src/commands/save.ts (6)
packages/cookiebench-cli/src/commands/results.ts (1)
RawBenchmarkDetail(37-177)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/shared/src/constants.ts (2)
HALF_SECOND(3-3)PERCENTAGE_DIVISOR(11-11)packages/cookiebench-cli/src/utils/scoring.ts (1)
calculateScores(1010-1379)
packages/cookiebench-cli/src/utils/index.ts (1)
packages/shared/src/utils/config.ts (1)
readConfig(14-26)
packages/cookiebench-cli/src/index.ts (9)
packages/cookiebench-cli/src/utils/logger.ts (3)
logger(152-152)CliLogger(8-8)createCliLogger(102-149)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)packages/shared/src/constants.ts (1)
HALF_SECOND(3-3)packages/cookiebench-cli/src/components/intro.ts (1)
displayIntro(11-93)packages/cookiebench-cli/src/commands/benchmark.ts (1)
benchmarkCommand(317-501)packages/cookiebench-cli/src/commands/results.ts (1)
resultsCommand(882-1112)packages/cookiebench-cli/src/commands/scores.ts (1)
scoresCommand(93-183)packages/cookiebench-cli/src/commands/save.ts (1)
saveCommand(228-391)packages/cookiebench-cli/src/commands/db.ts (1)
dbCommand(46-125)
packages/runner/src/server.ts (3)
packages/runner/src/types.ts (1)
ServerInfo(18-21)packages/shared/src/utils/package-manager.ts (1)
getPackageManager(5-37)packages/shared/src/constants.ts (1)
ONE_SECOND(2-2)
packages/cookiebench-cli/src/commands/results.ts (6)
packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/cookiebench-cli/src/types/index.ts (2)
Config(5-5)BenchmarkScores(10-38)packages/benchmark/src/types.ts (1)
Config(13-56)packages/shared/src/constants.ts (3)
KILOBYTE(7-7)PERCENTAGE_DIVISOR(11-11)ONE_SECOND(2-2)packages/cookiebench-cli/src/utils/constants.ts (16)
SCORE_THRESHOLD_POOR(10-10)SCORE_THRESHOLD_FAIR(11-11)CLS_DECIMAL_PLACES(7-7)CLS_THRESHOLD_GOOD(14-14)CLS_THRESHOLD_FAIR(15-15)COL_WIDTH_NAME(18-18)COL_WIDTH_CHART_PADDING(24-24)MAX_FILENAME_LENGTH(27-27)TRUNCATED_FILENAME_LENGTH(28-28)MIN_DURATION_THRESHOLD(31-31)COL_WIDTH_TYPE(19-19)COL_WIDTH_SOURCE(20-20)COL_WIDTH_SIZE(21-21)COL_WIDTH_DURATION(22-22)COL_WIDTH_TAGS(23-23)DEFAULT_DOM_SIZE(3-3)packages/cookiebench-cli/src/utils/scoring.ts (1)
calculateScores(1010-1379)
packages/runner/src/performance-aggregator.ts (3)
packages/benchmark/src/types.ts (7)
CoreWebVitals(214-225)CookieBannerData(93-101)CookieBannerMetrics(78-91)NetworkRequest(104-111)NetworkMetrics(113-116)ResourceTimingData(126-211)PerfumeMetrics(228-259)packages/runner/src/types.ts (9)
CoreWebVitals(10-10)CookieBannerData(8-8)CookieBannerMetrics(9-9)NetworkRequest(12-12)NetworkMetrics(11-11)ResourceTimingData(14-14)PerfumeMetrics(13-13)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/shared/src/constants.ts (2)
TTI_BUFFER_MS(14-14)PERCENTAGE_MULTIPLIER(10-10)
packages/shared/src/utils/conversion.ts (2)
packages/shared/src/index.ts (7)
bytesToKB(16-16)BYTES_TO_KB(5-5)formatBytes(18-18)KILOBYTE(7-7)decimalToPercentage(17-17)PERCENTAGE_MULTIPLIER(10-10)percentageToDecimal(19-19)packages/shared/src/constants.ts (3)
BYTES_TO_KB(6-6)KILOBYTE(7-7)PERCENTAGE_MULTIPLIER(10-10)
packages/cookiebench-cli/src/commands/scores.ts (6)
packages/cookiebench-cli/src/commands/results.ts (2)
BenchmarkOutput(179-213)RawBenchmarkDetail(37-177)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(152-152)CliLogger(8-8)packages/benchmark/src/types.ts (1)
Config(13-56)packages/shared/src/constants.ts (2)
HALF_SECOND(3-3)PERCENTAGE_DIVISOR(11-11)packages/cookiebench-cli/src/utils/scoring.ts (2)
printScores(1383-1422)calculateScores(1010-1379)
benchmarks/with-cookie-control/app/layout.tsx (2)
benchmarks/with-iubenda/app/layout.tsx (1)
metadata(4-6)benchmarks/with-onetrust/app/layout.tsx (1)
metadata(4-6)
packages/cookiebench-cli/src/utils/scoring.ts (3)
packages/shared/src/utils/time.ts (1)
formatTime(8-13)packages/shared/src/utils/conversion.ts (1)
formatBytes(17-25)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)
🪛 ast-grep (0.39.6)
benchmarks/with-iubenda/app/layout.tsx
[warning] 18-18: Usage of dangerouslySetInnerHTML detected. This bypasses React's built-in XSS protection. Always sanitize HTML content using libraries like DOMPurify before injecting it into the DOM to prevent XSS attacks.
Context: dangerouslySetInnerHTML
Note: [CWE-79] Improper Neutralization of Input During Web Page Generation [REFERENCES]
- https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml
- https://cwe.mitre.org/data/definitions/79.html
(react-unsafe-html-injection)
benchmarks/with-cookie-control/app/layout.tsx
[warning] 22-22: Usage of dangerouslySetInnerHTML detected. This bypasses React's built-in XSS protection. Always sanitize HTML content using libraries like DOMPurify before injecting it into the DOM to prevent XSS attacks.
Context: dangerouslySetInnerHTML
Note: [CWE-79] Improper Neutralization of Input During Web Page Generation [REFERENCES]
- https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml
- https://cwe.mitre.org/data/definitions/79.html
(react-unsafe-html-injection)
benchmarks/with-onetrust/app/layout.tsx
[warning] 23-23: Usage of dangerouslySetInnerHTML detected. This bypasses React's built-in XSS protection. Always sanitize HTML content using libraries like DOMPurify before injecting it into the DOM to prevent XSS attacks.
Context: dangerouslySetInnerHTML
Note: [CWE-79] Improper Neutralization of Input During Web Page Generation [REFERENCES]
- https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml
- https://cwe.mitre.org/data/definitions/79.html
(react-unsafe-html-injection)
🪛 LanguageTool
README.md
[grammar] ~7-~7: Ensure spelling is correct
Context: ...rics when loading web applications with diiferent cookie consent solutions. It helps deve...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🪛 markdownlint-cli2 (0.18.1)
packages/shared/README.md
22-22: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
27-27: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
31-31: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
37-37: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
40-40: Headings should be surrounded by blank lines
Expected: 1; Actual: 0; Below
(MD022, blanks-around-headings)
77-77: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
78-78: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
81-81: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
82-82: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
85-85: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
86-86: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
87-87: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
🔇 Additional comments (32)
benchmarks/with-cookie-control/app/layout.tsx (2)
1-2: Formatting consistency with double quotes approved.The quote changes align with the other layout components (iubenda, onetrust) and improve consistency across the benchmark apps.
Also applies to: 5-5
22-23: Security acknowledgment with biome-ignore comments is appropriate.The hardcoded script configuration is not user-provided or dynamic, so using
dangerouslySetInnerHTMLhere is reasonable and the biome-ignore comments properly document this decision. However, if this script configuration ever becomes dynamic or user-influenced in the future, this should be revisited and the content should be sanitized before injection.Verify that the injected script content remains static and hardcoded (not user-provided or dynamically generated from external sources).
Also applies to: 51-52
benchmarks/with-onetrust/app/layout.tsx (2)
16-21: LGTM: attribute reordering is cosmetic.The reordering of
asyncandsrcattributes has no runtime impact. This appears to be an auto-formatter adjustment.
22-25: LGTM: biome-ignore is appropriate for static content.The
biome-ignorecomment is valid here. The content being set ("function OptanonWrapper() {}") is a hardcoded function declaration required by OneTrust and poses no XSS risk. The static analysis warning is a false positive in this context.packages/shared/tsconfig.json (2)
4-5: Module strategy is consistent across all packages — no compatibility issues.Verification confirms that all consuming packages (benchmark, runner, cookiebench-cli) have identical module configurations to the shared package:
"module": "ESNext"with"moduleResolution": "bundler". No compatibility concerns exist.
1-14: No alignment issues found—configuration is consistent across all packages.The shared package's TypeScript configuration is properly aligned with packages/benchmark, packages/runner, and packages/cookiebench-cli. All packages consistently use the same module resolution strategy (
bundler), output target (ES2020/ESNext), and directory structure (src/dist). The configurations are complementary and compatible across the monorepo.packages/shared/rslib.config.ts (1)
1-19: No issues found. Configuration is appropriate for the project.Verification confirms all compatibility concerns are satisfied:
- ESM-only format: All consuming packages (
@consentio/runner,@consentio/benchmark,cookiebench) are ESM-based with"type": "module". No CommonJS consumers detected.- ES2021 syntax: Project requires Node.js >=18, which exceeds the ES2021 minimum of 14+.
The configuration is well-suited for this codebase and follows modern best practices.
benchmarks/with-iubenda/app/layout.tsx (1)
18-84: These changes appear unrelated to the PR objectives.This PR is titled "Refactored cli" and focuses on reorganizing benchmark CLI architecture, but the changes in this file (adding a linter suppression comment and reordering a script attribute) don't appear to relate to CLI refactoring. Were these changes included intentionally, or should they be part of a separate cleanup PR?
benchmarks/with-onetrust/config.json (1)
22-22: Correct typo in bundleType value.The value has been corrected from "iffe" to "iife" (Immediately Invoked Function Expression). This aligns with the bundle type corrections applied across multiple benchmark configurations.
README.md (1)
96-96: Update bundleType documentation.The documentation now correctly reflects the bundleType values ("iife", "cjs", "esm") following the standardized bundle type corrections across benchmark configurations.
.gitignore (1)
45-45: No issues found — addition of.pnpm-storeis correct and past problematic entries have been resolved.The verification confirms that:
.pnpm-storeis appropriately added to ignore pnpm's package store directory- The past review's concerns about
c15t/cli/andc15t/logger/entries have been fully resolved—no such entries exist in the current.gitignore- The file is clean and follows standard conventions
benchmarks/with-ketch/config.json (1)
19-19: LGTM! Typo correction aligns with schema and constants.The bundleType correction from "iffe" to "iife" properly matches the BUNDLE_TYPES.IIFE constant and JSON schema enum update, ensuring consistent bundle type detection across the benchmark suite.
packages/shared/src/utils/config.ts (1)
14-26: LGTM! Synchronous file read is appropriate for CLI config loading.The generic config reader implementation is well-suited for CLI startup configuration loading. The synchronous readFileSync is acceptable in this context, as config files are typically small and loaded once during application initialization.
packages/benchmark-schema/schema.json (1)
121-121: LGTM! Schema enum corrected to match constants.The bundleType enum correction from "iffe" to "iife" ensures schema validation aligns with the BUNDLE_TYPES constant definitions and configuration files across the codebase.
packages/shared/src/utils/conversion.ts (1)
8-43: LGTM! Clean conversion utilities with proper constants.All conversion functions are implemented correctly with appropriate use of imported constants. The formatBytes function follows standard logarithmic scaling for human-readable size formatting.
packages/shared/src/constants.ts (1)
1-14: LGTM! Constants provide clear semantic distinction.The constants are well-organized and appropriately named. While BYTES_TO_KB and KILOBYTE share the same value (1024), they serve distinct semantic purposes in conversion contexts, improving code readability.
packages/benchmark/src/constants.ts (1)
22-27: LGTM! Previous typo has been corrected.The BUNDLE_TYPES constant now correctly defines IIFE as "iife". This resolves the critical typo that was flagged in previous review comments and ensures proper bundle type detection throughout the benchmark suite.
benchmarks/with-osano/config.json (1)
16-16: LGTM! Typo correction aligns with schema and constants.The bundleType correction from "iffe" to "iife" properly matches the BUNDLE_TYPES.IIFE constant and JSON schema enum update, ensuring consistent bundle type detection across the benchmark suite.
benchmarks/with-didomi/package.json (1)
17-23: Verify missingcookiebenchdevDependency.This package is missing the
cookiebenchworkspace devDependency that was added to other benchmarks. Confirm this is intentional.packages/shared/src/utils/time.ts (1)
1-13: LGTM!The
formatTimeutility is well-implemented with clear logic, proper JSDoc, and correct formatting for both millisecond and second ranges.benchmarks/with-didomi/app/layout.tsx (1)
32-34: Good defensive programming.The null check prevents potential runtime errors if
onConsentChangedis called beforeonDidomiReadysets thedidomiObjectstate.packages/shared/package.json (1)
1-29: LGTM!The package manifest follows best practices with proper ESM configuration, TypeScript setup, and standard build tooling.
benchmarks/with-ketch/package.json (1)
16-22: Confirm whethercookiebenchdevDependency omission is intentional.Verification confirms that with-ketch (and also with-didomi) lack the direct
"cookiebench": "workspace:*"devDependency, while 10 other benchmarks include it. Both packages use the scoped@cookiebench/*packages but not the direct dependency. Clarify whether this omission is intentional or an oversight.packages/shared/src/index.ts (1)
1-22: LGTM!The barrel export file is well-organized with clear sections for constants and utilities. The biome ignore directive is appropriate for this use case.
packages/cookiebench-cli/src/utils/index.ts (1)
1-36: LGTM! Duplication eliminated via shared utilities.This implementation correctly addresses previous code duplication concerns by importing shared utilities from
@consentio/sharedand providing thin wrappers where CLI-specific behavior is needed. ThereadConfigfunction delegates toreadConfigShared<Config>, maintaining type safety while reusing the core implementation.packages/benchmark/package.json (1)
1-34: LGTM! Well-structured package manifest.The package configuration follows best practices with proper exports mapping, consistent build tooling, and appropriate dependency declarations. The setup aligns with the broader refactoring to extract benchmark functionality into a dedicated package.
packages/runner/package.json (1)
1-34: LGTM! Consistent package structure.The runner package manifest follows the same well-organized structure as the benchmark package, with proper workspace dependencies and build configuration. The dependency chain (runner → benchmark → shared) is clear and appropriate.
packages/runner/src/index.ts (1)
1-22: LGTM! Clean barrel export structure.The index provides a well-organized public API surface with clear grouping (runner, aggregation, server, types) and appropriate use of type-only exports. The structure facilitates clean imports for package consumers.
packages/cookiebench-cli/src/utils/constants.ts (1)
1-31: LGTM!The CLI-specific constants are well-organized and provide clear defaults for formatting, thresholds, and layout. The naming is consistent and self-documenting.
packages/cookiebench-cli/src/utils/scoring.ts (1)
1-1422: Well-structured scoring implementation.The scoring logic is comprehensive and well-organized with clear helper functions for each category (performance, bundle, network, transparency, UX). The addition of Perfume.js metrics (TTFB, INP) and network information enhances the scoring accuracy. The code properly handles edge cases with fallbacks and validation.
packages/cookiebench-cli/src/commands/benchmark.ts (2)
194-312: Solid benchmark execution flow with proper cleanup.The
runSingleBenchmarkfunction properly handles both local and remote benchmarking scenarios, uses a finally block to ensure server cleanup, and includes comprehensive error handling. The metric calculation and score computation are well-integrated.
317-501: Well-designed multi-benchmark orchestration.The command properly handles both single-app and multi-select flows, provides clear user feedback with progress indicators, and integrates the results panel seamlessly. The iteration override feature is well-implemented with per-config defaults.
| ✅ **@consentio/benchmark** - Fully integrated | ||
| - Uses shared constants for time and size conversions | ||
| - Imports directly via `@consentio/shared` | ||
|
|
||
| ✅ **@consentio/runner** - Fully integrated | ||
| - Re-exports shared utilities with proper Config typing | ||
| - Provides typed wrappers: `formatTime`, `getPackageManager`, `readConfig` | ||
|
|
||
| ✅ **cookiebench CLI** - Fully integrated | ||
| - Re-exports shared utilities and constants via `utils/index.ts` | ||
| - Provides typed `readConfig` wrapper for CLI Config type | ||
| - All duplicate implementations removed |
There was a problem hiding this comment.
Fix markdown list indentation.
The nested list items under "Integration Status" have incorrect indentation (2 spaces instead of 0), which may cause rendering issues.
Apply this diff to correct the indentation:
## Integration Status
✅ **@consentio/benchmark** - Fully integrated
- - Uses shared constants for time and size conversions
- - Imports directly via `@consentio/shared`
+- Uses shared constants for time and size conversions
+- Imports directly via `@consentio/shared`
✅ **@consentio/runner** - Fully integrated
- - Re-exports shared utilities with proper Config typing
- - Provides typed wrappers: `formatTime`, `getPackageManager`, `readConfig`
+- Re-exports shared utilities with proper Config typing
+- Provides typed wrappers: `formatTime`, `getPackageManager`, `readConfig`
✅ **cookiebench CLI** - Fully integrated
- - Re-exports shared utilities and constants via `utils/index.ts`
- - Provides typed `readConfig` wrapper for CLI Config type
- - All duplicate implementations removed
+- Re-exports shared utilities and constants via `utils/index.ts`
+- Provides typed `readConfig` wrapper for CLI Config type
+- All duplicate implementations removed🧰 Tools
🪛 markdownlint-cli2 (0.18.1)
77-77: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
78-78: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
81-81: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
82-82: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
85-85: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
86-86: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
87-87: Unordered list indentation
Expected: 0; Actual: 2
(MD007, ul-indent)
🤖 Prompt for AI Agents
In packages/shared/README.md around lines 76 to 87, the nested list items under
"Integration Status" are indented with two spaces which breaks markdown
rendering; remove the two-space indentation so each list item begins at the left
margin (no leading spaces) and keep the sub-bullets properly nested under their
parent entries, ensuring consistent use of a single space after the dash and
preserving the existing text.
| { | ||
| "compilerOptions": { | ||
| "target": "ES2020", | ||
| "module": "ESNext", | ||
| "moduleResolution": "bundler", | ||
| "esModuleInterop": true, | ||
| "strict": true, | ||
| "skipLibCheck": true, | ||
| "outDir": "dist", | ||
| "rootDir": "src" | ||
| }, | ||
| "include": ["src/**/*"], | ||
| "exclude": ["node_modules", "dist"] | ||
| } |
There was a problem hiding this comment.
Add declaration: true to emit TypeScript declaration files for consumer packages.
As a shared utilities package, emitting .d.ts files enables better type safety and IDE support for consuming packages (benchmark, runner, cookiebench-cli). This is essential for a monorepo where internal packages depend on shared types.
Apply this diff to enable TypeScript declaration emission:
{
"compilerOptions": {
"target": "ES2020",
"module": "ESNext",
"moduleResolution": "bundler",
"esModuleInterop": true,
"strict": true,
"skipLibCheck": true,
"outDir": "dist",
"rootDir": "src"
+ "declaration": true,
+ "declarationMap": true
},
"include": ["src/**/*"],
"exclude": ["node_modules", "dist"]
}Additionally, consider adding sourceMap: true for development debugging.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| { | |
| "compilerOptions": { | |
| "target": "ES2020", | |
| "module": "ESNext", | |
| "moduleResolution": "bundler", | |
| "esModuleInterop": true, | |
| "strict": true, | |
| "skipLibCheck": true, | |
| "outDir": "dist", | |
| "rootDir": "src" | |
| }, | |
| "include": ["src/**/*"], | |
| "exclude": ["node_modules", "dist"] | |
| } | |
| { | |
| "compilerOptions": { | |
| "target": "ES2020", | |
| "module": "ESNext", | |
| "moduleResolution": "bundler", | |
| "esModuleInterop": true, | |
| "strict": true, | |
| "skipLibCheck": true, | |
| "outDir": "dist", | |
| "rootDir": "src", | |
| "declaration": true, | |
| "declarationMap": true | |
| }, | |
| "include": ["src/**/*"], | |
| "exclude": ["node_modules", "dist"] | |
| } |
🤖 Prompt for AI Agents
In packages/shared/tsconfig.json lines 1-14, the tsconfig is missing the setting
to emit TypeScript declaration files and optional source maps; update
compilerOptions to include "declaration": true to produce .d.ts files for
consumers, and optionally add "sourceMap": true for development debugging (add
both under compilerOptions and keep outDir/rootDir as-is so declarations are
emitted into dist alongside compiled JS).
…g the first-party hostname from configuration or provided URL. Update performance metrics to handle null values for unmeasured data. Enhance logging utility to handle circular references. Restore '@types/figlet' dependency in package.json files for consistency across projects.
…e" to "engines" for consistency in Node.js version specification.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (5)
packages/cookiebench-cli/package.json (2)
10-10: Shebang configuration still required for CLI executability.The
binentry points todist/index.mjs, but rslib/Rspack won't auto-preserve the shebang line needed to make the CLI executable as a command-line tool. This issue was flagged in the previous review and still needs resolution.Add a postbuild script to ensure the output is executable:
"scripts": { "build": "rslib build", + "postbuild": "chmod +x ./dist/index.mjs", "check-types": "tsc --noEmit",Alternatively, configure rslib to inject the shebang via
BannerPlugininrslib.config.ts(not shown here).
22-22: Upgrade@clack/promptsfrom alpha to stable release.The dependency is pinned to
1.0.0-alpha.0, but a stable version0.11.0is available. Alpha versions risk breaking changes and stability issues in production. This issue was flagged in the previous review.Update to the stable release:
- "@clack/prompts": "^1.0.0-alpha.0", + "@clack/prompts": "^0.11.0",packages/cookiebench-cli/src/utils/logger.ts (2)
8-8: Unused export remains: removevalidLogLevelsor add usage.This was flagged in a previous review and remains unresolved. The
validLogLevelsconstant is exported but not consumed by any module in the codebase.Based on learnings.
132-134: Unresolved:messagemethod still drops variadic args.This issue was flagged in the previous review and remains unfixed. The
CliExtensionstype at line 13 declaresmessage: (message: string, ...args: unknown[]) => void, but the implementation only accepts(message: string)and drops any additional arguments, causing silent data loss.Apply the suggested fix from the previous review:
- extendedLogger.message = (message: string) => { - log.message(message); + extendedLogger.message = (message: string, ...args: unknown[]) => { + if (args.length === 0) { + log.message(message); + } else { + log.message(`${message}${formatArgs(args)}`); + } };Based on learnings.
packages/runner/src/performance-aggregator.ts (1)
228-266: Average Perfume metrics over actual samplesRight now we coerce missing FID/INP readings to 0 and divide by
results.length, so a series like[120, null, 80]reports66.7instead of the correct100. Also,firstPaint || nulldrops real 0 values. UseavgDefinedwith??to average over the defined samples and keep zeros intact.- const firstPaintValues = results.map((r) => r.timing.firstPaint || null); - const cookieBannerFirstPaint = avgDefined(firstPaintValues); + const firstPaintValues = results.map((r) => r.timing.firstPaint ?? null); + const cookieBannerFirstPaint = avgDefined(firstPaintValues); + const firstInputDelayAvg = avgDefined( + results.map((curr) => curr.timing.firstInputDelay ?? null) + ); + const interactionToNextPaintAvg = avgDefined( + results.map((curr) => curr.timing.interactionToNextPaint ?? null) + ); @@ - firstInputDelay: - results.reduce( - (acc, curr) => acc + (curr.timing.firstInputDelay || 0), - 0 - ) / results.length, - interactionToNextPaint: - results.reduce( - (acc, curr) => acc + (curr.timing.interactionToNextPaint || 0), - 0 - ) / results.length, + firstInputDelay: firstInputDelayAvg ?? 0, + interactionToNextPaint: interactionToNextPaintAvg ?? 0,
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (8)
packages/benchmark/src/network-monitor.ts(1 hunks)packages/cookiebench-cli/package.json(1 hunks)packages/cookiebench-cli/src/utils/logger.ts(1 hunks)packages/runner/src/benchmark-runner.ts(1 hunks)packages/runner/src/performance-aggregator.ts(1 hunks)packages/runner/src/server.ts(1 hunks)packages/runner/src/types.ts(1 hunks)packages/shared/src/utils/package-manager.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (6)
packages/benchmark/src/network-monitor.ts (2)
packages/benchmark/src/types.ts (3)
Config(13-56)NetworkRequest(104-111)NetworkMetrics(113-116)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(8-20)
packages/shared/src/utils/package-manager.ts (1)
packages/shared/src/index.ts (1)
getPackageManager(21-21)
packages/runner/src/server.ts (3)
packages/runner/src/types.ts (1)
ServerInfo(18-21)packages/shared/src/utils/package-manager.ts (1)
getPackageManager(5-38)packages/shared/src/constants.ts (1)
ONE_SECOND(2-2)
packages/runner/src/performance-aggregator.ts (3)
packages/runner/src/types.ts (10)
CoreWebVitals(10-10)CookieBannerData(8-8)CookieBannerMetrics(9-9)NetworkRequest(12-12)NetworkMetrics(11-11)ResourceTimingData(14-14)Config(6-6)PerfumeMetrics(13-13)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/benchmark/src/types.ts (8)
CoreWebVitals(214-225)CookieBannerData(93-101)CookieBannerMetrics(78-91)NetworkRequest(104-111)NetworkMetrics(113-116)ResourceTimingData(126-211)Config(13-56)PerfumeMetrics(228-259)packages/shared/src/constants.ts (2)
TTI_BUFFER_MS(14-14)PERCENTAGE_MULTIPLIER(10-10)
packages/runner/src/benchmark-runner.ts (8)
packages/runner/src/types.ts (3)
Config(6-6)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/benchmark/src/types.ts (1)
Config(13-56)packages/benchmark/src/cookie-banner-collector.ts (1)
CookieBannerCollector(13-242)packages/benchmark/src/network-monitor.ts (1)
NetworkMonitor(6-135)packages/benchmark/src/resource-timing-collector.ts (1)
ResourceTimingCollector(6-169)packages/benchmark/src/perfume-collector.ts (1)
PerfumeCollector(6-142)packages/runner/src/performance-aggregator.ts (1)
PerformanceAggregator(26-382)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(8-20)
packages/runner/src/types.ts (2)
packages/runner/src/index.ts (3)
ServerInfo(21-21)BenchmarkDetails(10-10)BenchmarkResult(11-11)packages/cookiebench-cli/src/types/index.ts (3)
ServerInfo(6-6)BenchmarkDetails(3-3)BenchmarkResult(4-4)
🔇 Additional comments (10)
packages/cookiebench-cli/package.json (3)
26-26: Verifycli-table3downgrade compatibility.The version was downgraded from
^0.6.5to^0.6.3. Confirm this change is intentional and that no features or fixes from 0.6.4–0.6.5 are required by the CLI.
33-33: Good:@types/figletmoved to devDependencies.Type stubs belong in
devDependenciessince they're only needed during development and build time. This fixes the earlier trivial issue.
23-25: Workspace packages and CLI utilities look appropriate.The addition of workspace dependencies (
@consentio/benchmark,@consentio/runner,@consentio/shared) and CLI formatting utilities (figlet,picocolors,pretty-ms) align well with the CLI refactoring goals described in the PR.Also applies to: 28-30
packages/cookiebench-cli/src/utils/logger.ts (1)
20-34: LGTM: Circular reference handling implemented.The
formatArgsfunction now properly handles circular references and serialization errors with a try/catch block andutil.inspectfallback, addressing the issue raised in the previous review.packages/runner/src/types.ts (6)
1-1: LGTM!The import of
ChildProcessis correctly used in theServerInfotype definition.
3-15: LGTM!Re-exporting types from the benchmark package is a good pattern for providing a unified API surface.
17-21: LGTM!The
ServerInfotype is straightforward and correctly typed for managing server processes.
158-271: LGTM!The
BenchmarkResulttype structure is comprehensive and well-designed, with appropriate handling of optional and nullable fields. Thescoresobject provides detailed performance analysis with clear categorization.
273-291: Consider whether internal types should be exported.The
EnhancedCookieBannerTimingandThirdPartyMetricstypes are used in the exportedBenchmarkDetailstype but are not themselves exported. While this works, it may limit external code that needs to construct or manipulate these nested structures independently.If these are strictly internal implementation details, the current approach is fine. Otherwise, consider exporting them for better composability.
24-156: Confirm the duplication is real and document the architectural intent.Your observation is correct. Verification reveals:
thirdPartyfields are identically duplicated:
timing.thirdPartyand root-levelthirdPartyboth receive the same object frombuildThirdPartyMetrics()with identical structure (lines 160–164 vs. 180–189)
cookieBannerfields represent different abstractions:
timing.cookieBannercontains 10 fields (detailed: renderStart, renderEnd, interactionStart, interactionEnd, layoutShift, detected, selector, serviceName, visibilityTime, viewportCoverage)- Root-level
cookieBannercontains 5 fields (summary: detected, selector, serviceName, visibilityTime, viewportCoverage)- Both derive from the same source but serve different purposes
The codebase uses both representations inconsistently—some modules access
timing.cookieBanner, others access root-levelcookieBanner; similarly forthirdParty. This creates maintenance risk and API confusion.Recommended resolution:
- For
thirdParty: Consolidate into a single field (likely at root-level, since it's already used there)- For
cookieBanner: Either document the summary/detailed distinction with JSDoc or consolidate if the subset is unnecessary- Update all consumers to use a consistent field path to avoid confusion
| baseline: boolean; | ||
| techStack: { | ||
| bundler: string; | ||
| bundleType: string | string[]; |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider normalizing bundleType to always use an array.
The bundleType: string | string[] union requires consumers to handle both single values and arrays. For consistency and simpler usage, consider always using string[].
Apply this diff to normalize the type:
- bundleType: string | string[];
+ bundleType: string[];If a single bundle type is common, consumers can still provide a single-element array: bundleType: ["esm"].
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| bundleType: string | string[]; | |
| bundleType: string[]; |
🤖 Prompt for AI Agents
In packages/runner/src/types.ts around line 163, the union type bundleType:
string | string[] should be normalized to bundleType: string[]; update the type
declaration to string[] and then update all call sites, default values and
destructurings across the codebase to expect an array (wrap single string usages
into single-element arrays, update tests and any runtime validation), and ensure
exported typings and any docs are updated accordingly.
…erformance metrics collection. Removed redundant network idle wait and context creation, ensuring a more efficient benchmarking process.
…ance metrics handling and streamline monitoring setup. Update types to ensure consistent data types and default values for unmeasured metrics. Introduce utility functions for better configuration management and package manager detection.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (5)
packages/runner/src/server.ts (1)
55-71: Kill the server process before throwing on startup failure.When the health check exhausts all retries at line 70, the error is thrown without terminating the spawned
serverProcess, leaving it running and leaking the port. This issue was previously flagged but remains unresolved in the current code.Apply this fix:
while (retries < maxRetries) { try { const response = await fetch(url); if (response.ok) { logger.success("Server is ready!"); return { serverProcess, url }; } } catch { // Ignore error and retry } await new Promise((resolve) => setTimeout(resolve, ONE_SECOND)); retries += 1; } + // Kill the spawned process before throwing + if (!serverProcess.killed) { + serverProcess.kill(); + } throw new Error("Server failed to start"); }packages/runner/src/benchmark-runner.ts (1)
52-59: Pass target hostname to network monitor for third-party detection.The
NetworkMonitor.setupMonitoringcall at line 53 does not receive the page's hostname, which is required to classify requests as first-party vs third-party. Without this, the third-party metrics (line 96-97) will not populate correctly, breaking the headline benchmarking functionality. This critical issue was flagged in previous reviews and remains unresolved.Apply this fix:
// Initialize collectors const collector = new PerformanceMetricsCollector(); const cookieBannerMetrics = this.cookieBannerCollector.initializeMetrics(); // Setup monitoring and detection - await this.networkMonitor.setupMonitoring(page); + await this.networkMonitor.setupMonitoring(page, url); await this.cookieBannerCollector.setupDetection(page); await this.perfumeCollector.setupPerfume(page); // Navigate to the page this.logger.debug(`Navigating to: ${url}`); await page.goto(url, { waitUntil: "networkidle" });packages/runner/src/performance-aggregator.ts (3)
107-111: Remove redundant fallback in division.Line 109 uses
(totalBlockingTime || 1)as a fallback even though line 108 already ensurestotalBlockingTime > 0via the ternary guard. The|| 1is redundant. This issue was flagged in previous reviews but remains unresolved.Apply this diff:
const percentageFromCookies = totalBlockingTime > 0 - ? (cookieBannerEstimate / (totalBlockingTime || 1)) * + ? (cookieBannerEstimate / totalBlockingTime) * PERCENTAGE_MULTIPLIER : 0;
150-162: Use nullish coalescing to preserve legitimate zero values.Lines 150-152 use logical OR (
||) for fallbacks, which incorrectly treats valid zero-valued metrics (e.g.,timeToFirstByte: 0,firstInputDelay: 0) as falsy and replaces them with defaults. This makes zero measurements indistinguishable from missing data. Line 153 has the same issue withnavigationTiming. Use nullish coalescing (??) instead. This issue was flagged in previous reviews but remains unresolved.Apply this fix:
- timeToFirstByte: perfumeMetrics?.timeToFirstByte || 0, - firstInputDelay: perfumeMetrics?.firstInputDelay || null, - interactionToNextPaint: perfumeMetrics?.interactionToNextPaint || null, - navigationTiming: perfumeMetrics?.navigationTiming || { + timeToFirstByte: perfumeMetrics?.timeToFirstByte ?? 0, + firstInputDelay: perfumeMetrics?.firstInputDelay ?? null, + interactionToNextPaint: perfumeMetrics?.interactionToNextPaint ?? null, + navigationTiming: perfumeMetrics?.navigationTiming ?? { timeToFirstByte: 0, domInteractive: 0, domContentLoadedEventStart: 0, domContentLoadedEventEnd: 0, domComplete: 0, loadEventStart: 0, loadEventEnd: 0, }, - networkInformation: perfumeMetrics?.networkInformation || undefined, + networkInformation: perfumeMetrics?.networkInformation ?? undefined,
241-323: Distinguish unmeasured metrics from zero measurements.Multiple metrics default to
0when not calculated:
- Line 241:
speedIndex: 0- Line 262:
domSize: 0- Lines 292-311: Various third-party counters
- Line 323:
cookieBannerTiming.firstPaint: 0Defaulting to
0conflates "not measured" with "measured as zero," which can mislead consumers of this data. Consider usingnullfor unimplemented metrics or computing actual averages where data exists (e.g.,firstPaintfromresults[].timing.firstPaint). This issue was flagged in previous reviews but remains unresolved.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (7)
packages/runner/package.json(1 hunks)packages/runner/src/benchmark-runner.ts(1 hunks)packages/runner/src/index.ts(1 hunks)packages/runner/src/performance-aggregator.ts(1 hunks)packages/runner/src/server.ts(1 hunks)packages/runner/src/types.ts(1 hunks)packages/runner/src/utils.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (5)
packages/runner/src/server.ts (2)
packages/runner/src/types.ts (1)
ServerInfo(18-21)packages/runner/src/utils.ts (1)
getPackageManager(25-57)
packages/runner/src/performance-aggregator.ts (3)
packages/shared/src/constants.ts (2)
TTI_BUFFER_MS(14-14)PERCENTAGE_MULTIPLIER(10-10)packages/runner/src/types.ts (10)
CoreWebVitals(10-10)CookieBannerData(8-8)CookieBannerMetrics(9-9)NetworkRequest(12-12)NetworkMetrics(11-11)ResourceTimingData(14-14)Config(6-6)PerfumeMetrics(13-13)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/benchmark/src/types.ts (8)
CoreWebVitals(214-225)CookieBannerData(93-101)CookieBannerMetrics(78-91)NetworkRequest(104-111)NetworkMetrics(113-116)ResourceTimingData(126-211)Config(13-56)PerfumeMetrics(228-259)
packages/runner/src/utils.ts (2)
packages/cookiebench-cli/src/utils/index.ts (1)
readConfig(34-36)packages/benchmark/src/types.ts (1)
Config(13-56)
packages/runner/src/benchmark-runner.ts (8)
packages/runner/src/types.ts (3)
Config(6-6)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/benchmark/src/types.ts (1)
Config(13-56)packages/benchmark/src/cookie-banner-collector.ts (1)
CookieBannerCollector(13-242)packages/benchmark/src/network-monitor.ts (1)
NetworkMonitor(6-135)packages/benchmark/src/resource-timing-collector.ts (1)
ResourceTimingCollector(6-169)packages/benchmark/src/perfume-collector.ts (1)
PerfumeCollector(6-142)packages/runner/src/performance-aggregator.ts (1)
PerformanceAggregator(29-372)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(8-20)
packages/runner/src/types.ts (2)
packages/runner/src/index.ts (3)
ServerInfo(21-21)BenchmarkDetails(10-10)BenchmarkResult(11-11)packages/cookiebench-cli/src/types/index.ts (3)
ServerInfo(6-6)BenchmarkDetails(3-3)BenchmarkResult(4-4)
🔇 Additional comments (2)
packages/runner/package.json (1)
1-33: LGTM!The package manifest is well-structured with appropriate module configuration, exports map, and dependencies for the runner orchestration layer.
packages/runner/src/index.ts (1)
1-24: LGTM!Clean barrel file providing a well-organized public API surface for the runner package.
| export async function getPackageManager(): Promise<{ | ||
| command: string; | ||
| args: string[]; | ||
| }> { | ||
| try { | ||
| const { execSync } = await import("node:child_process"); | ||
| const output = execSync("npm -v", { encoding: "utf-8" }); | ||
| if (output) { | ||
| return { command: "npm", args: ["run"] }; | ||
| } | ||
| } catch { | ||
| try { | ||
| const { execSync } = await import("node:child_process"); | ||
| const output = execSync("yarn -v", { encoding: "utf-8" }); | ||
| if (output) { | ||
| return { command: "yarn", args: [] }; | ||
| } | ||
| } catch { | ||
| try { | ||
| const { execSync } = await import("node:child_process"); | ||
| const output = execSync("pnpm -v", { encoding: "utf-8" }); | ||
| if (output) { | ||
| return { command: "pnpm", args: [] }; | ||
| } | ||
| } catch { | ||
| // Default to npm if no package manager is found | ||
| return { command: "npm", args: ["run"] }; | ||
| } | ||
| } | ||
| } | ||
| // Default to npm if no package manager is found | ||
| return { command: "npm", args: ["run"] }; | ||
| } |
There was a problem hiding this comment.
Optimize package manager detection.
The function has several issues:
- Redundant dynamic imports: Lines 30, 37, and 44 each dynamically import
child_processseparately, which is inefficient. - Unreachable code: Lines 55-56 return a default value, but this code is unreachable because lines 50-51 already return the same default in the innermost catch block.
- Silent error swallowing: Empty catch blocks at lines 35, 42, and 49 hide failures without logging.
Apply this refactor to import once and remove dead code:
export async function getPackageManager(): Promise<{
command: string;
args: string[];
}> {
+ const { execSync } = await import("node:child_process");
+
try {
- const { execSync } = await import("node:child_process");
const output = execSync("npm -v", { encoding: "utf-8" });
if (output) {
return { command: "npm", args: ["run"] };
}
} catch {
try {
- const { execSync } = await import("node:child_process");
const output = execSync("yarn -v", { encoding: "utf-8" });
if (output) {
return { command: "yarn", args: [] };
}
} catch {
try {
- const { execSync } = await import("node:child_process");
const output = execSync("pnpm -v", { encoding: "utf-8" });
if (output) {
return { command: "pnpm", args: [] };
}
} catch {
- // Default to npm if no package manager is found
- return { command: "npm", args: ["run"] };
+ // All checks failed, fall through to default
}
}
}
// Default to npm if no package manager is found
return { command: "npm", args: ["run"] };
}🤖 Prompt for AI Agents
In packages/runner/src/utils.ts around lines 25-57, replace the repeated dynamic
imports and nested try/catches by importing child_process.execSync once at the
start of the function, then sequentially attempt "npm -v", "yarn -v", and "pnpm
-v" each in its own small try/catch that does not swallow errors (log them with
console.debug or console.error), remove the redundant return in the innermost
catch and keep a single default return at the end of the function; this reduces
repeated imports, eliminates unreachable code, and surfaces failures instead of
silently swallowing them.
…ity files to improve code organization and maintainability. Add new lines for better readability and consistency across the project.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (2)
packages/shared/tsconfig.json (1)
2-11: Adddeclaration: trueanddeclarationMap: trueto emit TypeScript declaration files.This issue was flagged in the previous review but remains unresolved. As a shared utilities package in a monorepo, declaration files are essential for type safety and IDE support in consuming packages (benchmark, runner, cookiebench-cli).
Apply this diff to enable TypeScript declaration emission:
{ "compilerOptions": { "target": "ES2020", "module": "ESNext", "moduleResolution": "bundler", "esModuleInterop": true, "strict": true, "skipLibCheck": true, "outDir": "dist", "rootDir": "src", + "declaration": true, + "declarationMap": true, + "sourceMap": true }, "include": ["src/**/*"], "exclude": ["node_modules", "dist"] }This ensures
.d.tsfiles and source maps are emitted to thedistdirectory alongside compiled JavaScript, enabling proper type resolution across the monorepo.packages/shared/src/utils/package-manager.ts (1)
12-37: Apply the suggested refactoring to flatten nested try-catch blocks and eliminate repeated imports.The current code contains deeply nested error handling (3 levels) that imports
execSyncthree separate times and performs redundant output checks. The suggested refactoring uses a single import with loop-based iteration to improve readability, performance, and maintainability:export async function getPackageManager(): Promise< | { command: string; args: string[]; } | undefined > { + const { execSync } = await import("node:child_process"); + const managers = [ + { name: "npm", command: "npm", args: ["run"] }, + { name: "yarn", command: "yarn", args: [] }, + { name: "pnpm", command: "pnpm", args: [] }, + ]; + + for (const manager of managers) { + try { + execSync(`${manager.name} -v`, { encoding: "utf-8" }); + return { command: manager.command, args: manager.args }; + } catch { + // Try next manager + } + } + + // Default to npm if no package manager is found + return { command: "npm", args: ["run"] }; - try { - const { execSync } = await import("node:child_process"); - const output = execSync("npm -v", { encoding: "utf-8" }); - if (output) { - return { command: "npm", args: ["run"] }; - } - } catch { - try { - const { execSync } = await import("node:child_process"); - const output = execSync("yarn -v", { encoding: "utf-8" }); - if (output) { - return { command: "yarn", args: [] }; - } - } catch { - try { - const { execSync } = await import("node:child_process"); - const output = execSync("pnpm -v", { encoding: "utf-8" }); - if (output) { - return { command: "pnpm", args: [] }; - } - } catch { - // Default to npm if no package manager is found - return { command: "npm", args: ["run"] }; - } - } - } }
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (7)
packages/shared/package.json(1 hunks)packages/shared/rslib.config.ts(1 hunks)packages/shared/src/constants.ts(1 hunks)packages/shared/src/utils/config.ts(1 hunks)packages/shared/src/utils/package-manager.ts(1 hunks)packages/shared/src/utils/time.ts(1 hunks)packages/shared/tsconfig.json(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (4)
packages/shared/src/utils/config.ts (3)
packages/shared/src/index.ts (2)
BaseConfig(13-13)readConfig(13-13)packages/runner/src/index.ts (1)
readConfig(24-24)packages/cookiebench-cli/src/utils/index.ts (1)
readConfig(34-36)
packages/shared/src/utils/time.ts (4)
packages/runner/src/index.ts (1)
formatTime(24-24)packages/shared/src/index.ts (2)
formatTime(22-22)ONE_SECOND(8-8)packages/shared/src/constants.ts (1)
ONE_SECOND(2-2)packages/cookiebench-cli/src/utils/index.ts (1)
ONE_SECOND(22-22)
packages/shared/src/constants.ts (2)
packages/cookiebench-cli/src/utils/index.ts (5)
ONE_SECOND(22-22)HALF_SECOND(20-20)KILOBYTE(21-21)PERCENTAGE_MULTIPLIER(24-24)PERCENTAGE_DIVISOR(23-23)packages/shared/src/index.ts (7)
ONE_SECOND(8-8)HALF_SECOND(6-6)BYTES_TO_KB(5-5)KILOBYTE(7-7)PERCENTAGE_MULTIPLIER(10-10)PERCENTAGE_DIVISOR(9-9)TTI_BUFFER_MS(11-11)
packages/shared/src/utils/package-manager.ts (2)
packages/runner/src/index.ts (1)
getPackageManager(24-24)packages/shared/src/index.ts (1)
getPackageManager(21-21)
🔇 Additional comments (4)
packages/shared/rslib.config.ts (1)
1-19: No issues found. Configuration is compatible with monorepo Node.js requirements.The rslib configuration is appropriate for the shared package. Verification confirms all packages in the monorepo require Node.js >=18 (or higher), and ES2021 syntax has full support in Node.js 16+, so there are no compatibility concerns. The ESM-only format, TypeScript declarations, and Node target are well-suited for a modern shared package.
packages/shared/src/utils/time.ts (1)
1-13: LGTM! Clean and well-documented time formatting utility.The implementation correctly formats milliseconds with no decimals and seconds with two decimals, using the shared constant appropriately.
packages/shared/src/utils/config.ts (1)
1-26: LGTM! Appropriate generic config reader.The synchronous file read is acceptable for config files, and the null-returning error handling is clearly documented. The generic type parameter provides flexibility for consumers to specify their own config shapes.
packages/shared/package.json (1)
1-29: Package manifest structure looks good.The exports, scripts, and devDependencies are properly configured for an ESM TypeScript library with workspace-based development dependencies.
| export const BYTES_TO_KB = 1024; | ||
| export const KILOBYTE = 1024; |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider consolidating duplicate constants.
BYTES_TO_KB and KILOBYTE both have the value 1024. Having two names for the same constant can be confusing. Consider using a single well-named constant unless these are intended to represent semantically different concepts.
If they're meant to be the same, consolidate to one:
// Size constants (bytes to kilobytes)
-export const BYTES_TO_KB = 1024;
-export const KILOBYTE = 1024;
+export const KILOBYTE = 1024;
+export const BYTES_TO_KB = KILOBYTE; // Alias for clarity in conversion contexts📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| export const BYTES_TO_KB = 1024; | |
| export const KILOBYTE = 1024; | |
| export const KILOBYTE = 1024; | |
| export const BYTES_TO_KB = KILOBYTE; // Alias for clarity in conversion contexts |
🤖 Prompt for AI Agents
In packages/shared/src/constants.ts around lines 6 to 7, there are duplicate
constants BYTES_TO_KB and KILOBYTE both set to 1024; consolidate them by keeping
a single well-named constant (e.g., KILOBYTE or BYTES_TO_KB) and replace all
usages of the removed name across the codebase with the chosen name, then remove
the redundant declaration and run tests/lint to ensure no references remain.
| export const PERCENTAGE_MULTIPLIER = 100; | ||
| export const PERCENTAGE_DIVISOR = 100; |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Evaluate the need for identical percentage constants.
PERCENTAGE_MULTIPLIER and PERCENTAGE_DIVISOR both equal 100. If multiplication and division operations conceptually differ in your domain, document why they share the same value. Otherwise, a single constant would reduce confusion.
🤖 Prompt for AI Agents
In packages/shared/src/constants.ts around lines 10 to 11, PERCENTAGE_MULTIPLIER
and PERCENTAGE_DIVISOR are both set to 100 which is confusing; either collapse
them into a single exported constant (e.g., PERCENTAGE_SCALE or PERCENTAGE_BASE
= 100) and update all imports/usages to that single name, or keep both but add a
short comment above the declarations explaining the conceptual difference and
why they share the same value; ensure you update any tests/consumers to use the
new name or keep backward-compatible re-exports if removing one constant.
…n opacity, improving UX metrics. Update types to include new visibility metrics and refine benchmark runner with robust error handling and retry logic. Introduce statistical utilities for performance analysis, ensuring better data handling and stability checks in performance aggregation.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 10
♻️ Duplicate comments (7)
packages/cookiebench-cli/src/commands/results.ts (1)
1050-1055: Fix third-party request counting to include all resource types.The previous review comment on this code segment is still valid.
thirdPartyRequestscurrently only counts script resources, ignoring third-party styles, images, fonts, and "other" assets. This underreports the network impact and leads to incorrect scores.Apply the diff from the previous review:
- thirdPartyRequests: - appResults.reduce( - (a, b) => - a + b.resources.scripts.filter((s) => s.isThirdParty).length, - 0 - ) / appResults.length, + thirdPartyRequests: + appResults.reduce( + (a, b) => + a + + b.resources.scripts.filter((r) => r.isThirdParty).length + + b.resources.styles.filter((r) => r.isThirdParty).length + + b.resources.images.filter((r) => r.isThirdParty).length + + b.resources.fonts.filter((r) => r.isThirdParty).length + + b.resources.other.filter((r) => r.isThirdParty).length, + 0 + ) / appResults.length,packages/cookiebench-cli/src/commands/scores.ts (1)
211-318: Guard against empty benchmark data before averaging.When
result.resultsis[](e.g., aborted run wrote an empty array), every reduction below divides by0, producingNaNscores and breaking the CLI output. Bail out before callingcalculateScoresso we never average an empty set.logger.debug("Calculating scores from raw benchmark data"); const appResults = result.results; + if (!Array.isArray(appResults) || appResults.length === 0) { + logger.error(`No iterations recorded for ${appName}; cannot compute scores.`); + return; + } const config = await loadConfigForApp(logger, appName);packages/runner/src/benchmark-runner.ts (1)
115-117: Pass the benchmark URL into NetworkMonitor.Right now
setupMonitoringonly receives thePage, so insideNetworkMonitorthe first-party hostname defaults topage.url(), which is stillabout:blankbefore navigation. Every request is therefore marked third-party and the aggregated third-party metrics inPerformanceAggregatorare garbage. Hand the actual benchmark URL through so classification stays correct.Apply this diff:
- await this.networkMonitor.setupMonitoring(page); + await this.networkMonitor.setupMonitoring(page, url);packages/runner/src/performance-aggregator.ts (4)
21-22: Constants still duplicated from shared package.Despite the previous review marking this as addressed,
TTI_BUFFER_MSandPERCENTAGE_MULTIPLIERremain defined here but already exist inpackages/shared/src/constants.ts.Based on relevant snippets.
118-122: Redundant fallback still present.Line 120 uses
(totalBlockingTime || 1)but the ternary on line 119 already ensures division only occurs whentotalBlockingTime > 0, making the|| 1fallback unnecessary.
162-163: Zero-value metrics still incorrectly converted to null.Using
|| nullconverts legitimate 0ms FID/INP values tonull. A First Input Delay or Interaction to Next Paint of 0ms is a valid measurement indicating no delay, not missing data.
290-347: Unimplemented metrics still default to zero.Multiple metrics default to
0when not tracked:
- Line 290:
speedIndex- Line 301:
domSize- Lines 309-328: All third-party detail metrics
- Lines 329-338:
thirdPartyTimingsub-fields- Line 340:
cookieBannerTiming.firstPaintDefaulting to
0conflates "not measured" with an actual zero value. While pragmatic for unimplemented features, consider usingnullor explicitly marking these as unavailable to distinguish missing data from zero measurements.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (9)
packages/benchmark/src/cookie-banner-collector.ts(1 hunks)packages/benchmark/src/types.ts(1 hunks)packages/cookiebench-cli/src/commands/benchmark.ts(1 hunks)packages/cookiebench-cli/src/commands/results.ts(1 hunks)packages/cookiebench-cli/src/commands/scores.ts(1 hunks)packages/runner/src/benchmark-runner.ts(1 hunks)packages/runner/src/index.ts(1 hunks)packages/runner/src/performance-aggregator.ts(1 hunks)packages/runner/src/statistics.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (8)
packages/benchmark/src/types.ts (4)
packages/runner/src/index.ts (9)
CookieBannerConfig(14-14)Config(13-13)CookieBannerMetrics(16-16)CookieBannerData(15-15)NetworkRequest(19-19)NetworkMetrics(18-18)BundleStrategy(12-12)ResourceTimingData(20-20)CoreWebVitals(17-17)packages/runner/src/types.ts (10)
CookieBannerConfig(7-7)Config(6-6)CookieBannerMetrics(9-9)CookieBannerData(8-8)NetworkRequest(12-12)NetworkMetrics(11-11)BundleStrategy(5-5)ResourceTimingData(14-14)CoreWebVitals(10-10)PerfumeMetrics(13-13)packages/benchmark/src/index.ts (13)
CookieBannerConfig(15-15)Config(14-14)LayoutShiftEntry(19-19)WindowWithCookieMetrics(24-24)CookieBannerMetrics(17-17)CookieBannerData(16-16)NetworkRequest(21-21)NetworkMetrics(20-20)BundleStrategy(13-13)ResourceTimingData(23-23)CoreWebVitals(18-18)PerfumeMetrics(22-22)WindowWithPerfumeMetrics(25-25)packages/cookiebench-cli/src/types/index.ts (1)
Config(5-5)
packages/runner/src/statistics.ts (1)
packages/runner/src/index.ts (4)
calculateStatistics(26-26)calculateTrimmedMean(27-27)calculateCoefficientOfVariation(25-25)isStable(28-28)
packages/cookiebench-cli/src/commands/scores.ts (6)
packages/cookiebench-cli/src/commands/results.ts (2)
BenchmarkOutput(179-213)RawBenchmarkDetail(37-177)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(162-162)CliLogger(9-9)packages/benchmark/src/types.ts (1)
Config(13-56)packages/shared/src/constants.ts (2)
HALF_SECOND(3-3)PERCENTAGE_DIVISOR(11-11)packages/cookiebench-cli/src/utils/scoring.ts (2)
printScores(1383-1422)calculateScores(1010-1379)
packages/benchmark/src/cookie-banner-collector.ts (3)
packages/benchmark/src/types.ts (5)
Config(13-56)CookieBannerMetrics(79-92)WindowWithCookieMetrics(65-77)LayoutShiftEntry(59-62)CookieBannerData(94-103)packages/benchmark/src/bundle-strategy.ts (1)
determineBundleStrategy(4-21)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(8-20)
packages/cookiebench-cli/src/commands/benchmark.ts (10)
packages/runner/src/index.ts (6)
BenchmarkResult(11-11)readConfig(31-31)ServerInfo(21-21)buildAndServeNextApp(7-7)BenchmarkRunner(3-3)cleanupServer(7-7)packages/runner/src/types.ts (2)
BenchmarkResult(158-271)ServerInfo(18-21)packages/cookiebench-cli/src/utils/constants.ts (4)
DEFAULT_THIRD_PARTY_DOMAINS(4-4)DEFAULT_DOM_SIZE(3-3)DEFAULT_ITERATIONS(2-2)SEPARATOR_WIDTH(5-5)packages/shared/src/constants.ts (2)
PERCENTAGE_DIVISOR(11-11)HALF_SECOND(3-3)packages/cookiebench-cli/src/utils/index.ts (3)
PERCENTAGE_DIVISOR(23-23)readConfig(34-36)HALF_SECOND(20-20)packages/runner/src/benchmark-runner.ts (2)
runSingleBenchmark(91-194)BenchmarkRunner(25-432)packages/shared/src/utils/config.ts (1)
readConfig(14-26)packages/runner/src/server.ts (2)
buildAndServeNextApp(6-71)cleanupServer(73-77)packages/cookiebench-cli/src/utils/scoring.ts (2)
calculateScores(1010-1379)printScores(1383-1422)packages/cookiebench-cli/src/commands/results.ts (1)
resultsCommand(890-1120)
packages/runner/src/benchmark-runner.ts (8)
packages/benchmark/src/types.ts (1)
Config(13-56)packages/runner/src/types.ts (3)
Config(6-6)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/benchmark/src/cookie-banner-collector.ts (1)
CookieBannerCollector(17-310)packages/benchmark/src/network-monitor.ts (1)
NetworkMonitor(6-135)packages/benchmark/src/resource-timing-collector.ts (1)
ResourceTimingCollector(6-169)packages/benchmark/src/perfume-collector.ts (1)
PerfumeCollector(6-142)packages/runner/src/performance-aggregator.ts (1)
PerformanceAggregator(38-446)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(8-20)
packages/cookiebench-cli/src/commands/results.ts (6)
packages/cookiebench-cli/src/utils/logger.ts (2)
logger(162-162)CliLogger(9-9)packages/benchmark/src/types.ts (1)
Config(13-56)packages/cookiebench-cli/src/types/index.ts (2)
Config(5-5)BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/constants.ts (16)
SCORE_THRESHOLD_POOR(10-10)SCORE_THRESHOLD_FAIR(11-11)CLS_DECIMAL_PLACES(7-7)CLS_THRESHOLD_GOOD(14-14)CLS_THRESHOLD_FAIR(15-15)COL_WIDTH_NAME(18-18)COL_WIDTH_CHART_PADDING(24-24)MAX_FILENAME_LENGTH(27-27)TRUNCATED_FILENAME_LENGTH(28-28)MIN_DURATION_THRESHOLD(31-31)COL_WIDTH_TYPE(19-19)COL_WIDTH_SOURCE(20-20)COL_WIDTH_SIZE(21-21)COL_WIDTH_DURATION(22-22)COL_WIDTH_TAGS(23-23)DEFAULT_DOM_SIZE(3-3)packages/cookiebench-cli/src/utils/scoring.ts (1)
calculateScores(1010-1379)packages/cookiebench-cli/src/utils/auth.ts (1)
isAdminUser(5-11)
packages/runner/src/performance-aggregator.ts (4)
packages/shared/src/constants.ts (2)
TTI_BUFFER_MS(14-14)PERCENTAGE_MULTIPLIER(10-10)packages/benchmark/src/types.ts (8)
CoreWebVitals(216-227)CookieBannerData(94-103)CookieBannerMetrics(79-92)NetworkRequest(106-113)NetworkMetrics(115-118)ResourceTimingData(128-213)Config(13-56)PerfumeMetrics(230-261)packages/runner/src/types.ts (10)
CoreWebVitals(10-10)CookieBannerData(8-8)CookieBannerMetrics(9-9)NetworkRequest(12-12)NetworkMetrics(11-11)ResourceTimingData(14-14)Config(6-6)PerfumeMetrics(13-13)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/runner/src/statistics.ts (4)
isStable(132-138)calculateCoefficientOfVariation(112-127)calculateTrimmedMean(84-107)calculateStatistics(11-52)
🔇 Additional comments (6)
packages/cookiebench-cli/src/commands/results.ts (2)
36-213: Well-structured type definitions.The
RawBenchmarkDetailandBenchmarkOutputtypes are comprehensive and properly model the benchmark data structure with appropriate optional fields.
1091-1091: Document why networkInformation uses only the first result.The
networkInformationis passed from only the first result. If this is intentional (because network conditions are assumed consistent across iterations), add a comment explaining the rationale. Otherwise, consider selecting the most representative value or aggregating connection quality metrics.packages/cookiebench-cli/src/commands/benchmark.ts (3)
103-157: Excellent handling of banner detection consistency.The implementation properly validates banner detection across all iterations and provides appropriate warnings when detection is inconsistent. The null/zero timing checks and coverage calculations are thorough.
298-303: Good use of finally block for cleanup.The server cleanup is correctly placed in a finally block, ensuring the local server is terminated even if benchmark execution fails.
384-387: Misleading comment about "most common iteration count".The comment claims to find the "most common iteration count" but the code simply takes the first value from the Map. Either fix the logic to find the actual mode or update the comment.
Apply this diff to fix the comment:
- // Find the most common iteration count or first one + // Use the first benchmark's iteration count as default const defaultIterations = benchmarkConfigs.size > 0 ? Array.from(benchmarkConfigs.values())[0] : DEFAULT_ITERATIONS;Or if you want the actual most common value:
- // Find the most common iteration count or first one + // Find the most common iteration count const defaultIterations = - benchmarkConfigs.size > 0 - ? Array.from(benchmarkConfigs.values())[0] - : DEFAULT_ITERATIONS; + benchmarkConfigs.size > 0 + ? Array.from(benchmarkConfigs.values()) + .sort( + (a, b) => + benchmarkConfigs + .values() + .toArray() + .filter((v) => v === b).length - + benchmarkConfigs + .values() + .toArray() + .filter((v) => v === a).length + )[0] + : DEFAULT_ITERATIONS;Likely an incorrect or invalid review comment.
packages/runner/src/performance-aggregator.ts (1)
354-445: LGTM: Statistical summary and logging methods.The
getStatisticalSummary,logResults, andlogStatisticalSummarymethods are well-structured. Statistical calculations delegate appropriately to the imported functions, logging provides useful debug and info output, and stability indicators give clear feedback on metric variability.
| if (results[data.app]) { | ||
| logger.warn( | ||
| `Duplicate app name "${data.app}" found in ${file}. Previous results will be overwritten.` | ||
| ); | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider preventing duplicate overwrites.
When duplicate app names are detected, the current implementation warns but proceeds to overwrite the previous results. This silently discards data, which could mask configuration errors or unexpected file structures.
Consider either:
- Throwing an error to halt execution and force resolution
- Merging results arrays instead of replacing them
- Using file paths as unique keys to preserve all data
🤖 Prompt for AI Agents
In packages/cookiebench-cli/src/commands/results.ts around lines 342 to 346, the
code only warns on duplicate app names and overwrites prior results; instead
preserve data by merging results arrays: detect if results[data.app] exists and
is an array, then concat/merge the incoming data results into the existing array
(and log an info message about merging), ensuring you don't lose previous
entries; also add a unit test or guard to validate the merged structure.
Optionally, if you prefer fail-fast behavior, replace the warning with throwing
an Error to halt execution or switch the key to a unique identifier (e.g.,
include source file path) to avoid collisions—implement one of these fixes and
remove the silent overwrite.
| function printDetailedResults( | ||
| appName: string, | ||
| results: RawBenchmarkDetail[], | ||
| scores: BenchmarkScores, | ||
| baseline?: RawBenchmarkDetail[] | ||
| ) { | ||
| console.log( | ||
| `\n${color.bold(color.cyan(`━━━ ${appName.toUpperCase()} ━━━`))}` | ||
| ); | ||
|
|
||
| // ━━━ Score Display ━━━ | ||
| const score = Math.round(scores.totalScore); | ||
| let scoreColor = color.green; | ||
| let scoreBgColor = color.bgGreen; | ||
|
|
||
| if (score < SCORE_THRESHOLD_POOR) { | ||
| scoreColor = color.red; | ||
| scoreBgColor = color.bgRed; | ||
| } else if (score < SCORE_THRESHOLD_FAIR) { | ||
| scoreColor = color.yellow; | ||
| scoreBgColor = color.bgYellow; | ||
| } | ||
|
|
||
| console.log(`\n${color.bold("🎯 Overall Score")}`); | ||
| console.log( | ||
| scoreColor(` ${score}/100`) + | ||
| " " + | ||
| scoreBgColor(color.black(` ${scores.grade} `)) | ||
| ); | ||
|
|
||
| // ━━━ Key Insights ━━━ | ||
| if (scores.insights && scores.insights.length > 0) { | ||
| console.log(`\n${color.bold("💡 Key Insights")}`); | ||
| for (const insight of scores.insights) { | ||
| console.log(`${color.blue(" •")} ${color.dim(insight)}`); | ||
| } | ||
| } | ||
|
|
||
| // Calculate averages | ||
| const avgBannerVisibility = | ||
| results.reduce((a, b) => a + b.timing.cookieBanner.visibilityTime, 0) / | ||
| results.length; | ||
| const avgViewportCoverage = | ||
| results.reduce((a, b) => a + b.timing.cookieBanner.viewportCoverage, 0) / | ||
| results.length; | ||
| const avgNetworkImpact = | ||
| results.reduce((a, b) => a + b.size.thirdParty, 0) / results.length; | ||
| const _bannerDetected = results.some((r) => r.timing.cookieBanner.detected); | ||
| const isBundled = results[0]?.size.thirdParty === 0; | ||
|
|
||
| const avgFCP = | ||
| results.reduce((a, b) => a + b.timing.firstContentfulPaint, 0) / | ||
| results.length; | ||
| const avgLCP = | ||
| results.reduce((a, b) => a + b.timing.largestContentfulPaint, 0) / | ||
| results.length; | ||
| const avgTTI = | ||
| results.reduce((a, b) => a + b.timing.timeToInteractive, 0) / | ||
| results.length; | ||
| const avgCLS = | ||
| results.reduce((a, b) => a + b.timing.cumulativeLayoutShift, 0) / | ||
| results.length; | ||
| const avgTBT = | ||
| results.reduce((a, b) => a + b.timing.mainThreadBlocking.total, 0) / | ||
| results.length; | ||
|
|
||
| const totalSize = | ||
| results.reduce((a, b) => a + b.size.total, 0) / results.length; | ||
| const jsSize = | ||
| results.reduce((a, b) => a + b.size.scripts.total, 0) / results.length; | ||
| const cssSize = | ||
| results.reduce((a, b) => a + b.size.styles, 0) / results.length; | ||
| const imageSize = | ||
| results.reduce((a, b) => a + b.size.images, 0) / results.length; | ||
| const fontSize = | ||
| results.reduce((a, b) => a + b.size.fonts, 0) / results.length; | ||
| const otherSize = | ||
| results.reduce((a, b) => a + b.size.other, 0) / results.length; | ||
|
|
||
| const jsFiles = | ||
| results.reduce((a, b) => a + b.resources.scripts.length, 0) / | ||
| results.length; | ||
| const cssFiles = | ||
| results.reduce((a, b) => a + b.resources.styles.length, 0) / results.length; | ||
| const imageFiles = | ||
| results.reduce((a, b) => a + b.resources.images.length, 0) / results.length; | ||
| const fontFiles = | ||
| results.reduce((a, b) => a + b.resources.fonts.length, 0) / results.length; | ||
| const otherFiles = | ||
| results.reduce((a, b) => a + b.resources.other.length, 0) / results.length; | ||
|
|
||
| // Calculate deltas if baseline exists | ||
| let bannerDelta = ""; | ||
| if (baseline && appName !== "baseline") { | ||
| const baselineAvgBanner = | ||
| baseline.reduce((a, b) => a + b.timing.cookieBanner.visibilityTime, 0) / | ||
| baseline.length; | ||
| const delta = avgBannerVisibility - baselineAvgBanner; | ||
| bannerDelta = ` ${delta > 0 ? "+" : ""}${formatTime(delta)}`; | ||
| } | ||
|
|
||
| // ━━━ Cookie Banner Impact ━━━ | ||
| console.log(`\n${color.bold("🍪 Cookie Banner Impact")}`); | ||
| const bannerTable = new Table({ | ||
| chars: { mid: "", "left-mid": "", "mid-mid": "", "right-mid": "" }, | ||
| style: { "padding-left": 2, "padding-right": 2, border: ["grey"] }, | ||
| }); | ||
|
|
||
| bannerTable.push( | ||
| [ | ||
| { content: "Banner Visibility", colSpan: 1 }, | ||
| { content: "Viewport Coverage", colSpan: 1 }, | ||
| { content: "Network Impact", colSpan: 1 }, | ||
| { content: "Bundle Strategy", colSpan: 1 }, | ||
| ], | ||
| [ | ||
| `${color.bold(formatTime(avgBannerVisibility))}\n${color.dim(bannerDelta || "baseline")}`, | ||
| `${color.bold(`${avgViewportCoverage.toFixed(1)}%`)}\n${color.dim("Screen real estate")}`, | ||
| `${color.bold(formatBytes(avgNetworkImpact * KILOBYTE))}\n${color.dim(isBundled ? "Bundled (no network)" : "External requests")}`, | ||
| `${color.bold(isBundled ? "Bundled" : "External")}\n${color.dim(isBundled ? "Included in main bundle" : "Loaded from CDN")}`, | ||
| ] | ||
| ); | ||
|
|
||
| console.log(bannerTable.toString()); | ||
|
|
||
| // ━━━ Core Web Vitals ━━━ | ||
| console.log(`\n${color.bold("⚡ Core Web Vitals")}`); | ||
| const vitalsTable = new Table({ | ||
| chars: { mid: "", "left-mid": "", "mid-mid": "", "right-mid": "" }, | ||
| style: { "padding-left": 2, "padding-right": 2, border: ["grey"] }, | ||
| }); | ||
|
|
||
| vitalsTable.push( | ||
| [ | ||
| { content: "First Contentful Paint", colSpan: 1 }, | ||
| { content: "Largest Contentful Paint", colSpan: 1 }, | ||
| { content: "Time to Interactive", colSpan: 1 }, | ||
| { content: "Cumulative Layout Shift", colSpan: 1 }, | ||
| ], | ||
| [ | ||
| `${color.bold(formatTime(avgFCP))}\n${getPerformanceRating("fcp", avgFCP)}`, | ||
| `${color.bold(formatTime(avgLCP))}\n${getPerformanceRating("lcp", avgLCP)}`, | ||
| `${color.bold(formatTime(avgTTI))}\n${getPerformanceRating("tti", avgTTI)}`, | ||
| `${color.bold(avgCLS.toFixed(CLS_DECIMAL_PLACES))}\n${getPerformanceRating("cls", avgCLS)}`, | ||
| ] | ||
| ); | ||
|
|
||
| console.log(vitalsTable.toString()); | ||
|
|
||
| // ━━━ Resource Breakdown ━━━ | ||
| console.log(`\n${color.bold("📦 Resource Breakdown")}`); | ||
|
|
||
| const totalFiles = jsFiles + cssFiles + imageFiles + fontFiles + otherFiles; | ||
| const jsPercentage = | ||
| totalSize > 0 ? (jsSize / totalSize) * PERCENTAGE_DIVISOR : 0; | ||
| const cssPercentage = | ||
| totalSize > 0 ? (cssSize / totalSize) * PERCENTAGE_DIVISOR : 0; | ||
| const imagePercentage = | ||
| totalSize > 0 ? (imageSize / totalSize) * PERCENTAGE_DIVISOR : 0; | ||
| const fontPercentage = | ||
| totalSize > 0 ? (fontSize / totalSize) * PERCENTAGE_DIVISOR : 0; | ||
| const otherPercentage = | ||
| totalSize > 0 ? (otherSize / totalSize) * PERCENTAGE_DIVISOR : 0; | ||
|
|
||
| const resourceTable = new Table({ | ||
| chars: { mid: "", "left-mid": "", "mid-mid": "", "right-mid": "" }, | ||
| style: { "padding-left": 2, "padding-right": 2, border: ["grey"] }, | ||
| }); | ||
|
|
||
| resourceTable.push( | ||
| [ | ||
| { content: "Type", colSpan: 1 }, | ||
| { content: "Size", colSpan: 1 }, | ||
| { content: "Files", colSpan: 1 }, | ||
| { content: "% of Total", colSpan: 1 }, | ||
| ], | ||
| [ | ||
| color.cyan("JavaScript"), | ||
| formatBytes(jsSize * KILOBYTE), | ||
| Math.round(jsFiles).toString(), | ||
| `${jsPercentage.toFixed(1)}%`, | ||
| ], | ||
| [ | ||
| color.cyan("CSS"), | ||
| formatBytes(cssSize * KILOBYTE), | ||
| Math.round(cssFiles).toString(), | ||
| `${cssPercentage.toFixed(1)}%`, | ||
| ], | ||
| [ | ||
| color.cyan("Images"), | ||
| formatBytes(imageSize * KILOBYTE), | ||
| Math.round(imageFiles).toString(), | ||
| `${imagePercentage.toFixed(1)}%`, | ||
| ], | ||
| [ | ||
| color.cyan("Fonts"), | ||
| formatBytes(fontSize * KILOBYTE), | ||
| Math.round(fontFiles).toString(), | ||
| `${fontPercentage.toFixed(1)}%`, | ||
| ], | ||
| [ | ||
| color.cyan("Other"), | ||
| formatBytes(otherSize * KILOBYTE), | ||
| Math.round(otherFiles).toString(), | ||
| `${otherPercentage.toFixed(1)}%`, | ||
| ], | ||
| [ | ||
| color.bold("Total"), | ||
| color.bold(formatBytes(totalSize * KILOBYTE)), | ||
| color.bold(Math.round(totalFiles).toString()), | ||
| color.bold("100%"), | ||
| ] | ||
| ); | ||
|
|
||
| console.log(resourceTable.toString()); | ||
|
|
||
| // ━━━ Performance Impact Summary ━━━ | ||
| console.log(`\n${color.bold("📊 Performance Impact Summary")}`); | ||
| const summaryTable = new Table({ | ||
| chars: { mid: "", "left-mid": "", "mid-mid": "", "right-mid": "" }, | ||
| style: { "padding-left": 2, "padding-right": 2, border: ["grey"] }, | ||
| }); | ||
|
|
||
| let layoutStability = "Poor"; | ||
| if (avgCLS === 0) { | ||
| layoutStability = "Perfect"; | ||
| } else if (avgCLS < CLS_THRESHOLD_GOOD) { | ||
| layoutStability = "Good"; | ||
| } else if (avgCLS < CLS_THRESHOLD_FAIR) { | ||
| layoutStability = "Fair"; | ||
| } | ||
|
|
||
| summaryTable.push( | ||
| ["Loading Strategy", color.bold(isBundled ? "Bundled" : "External")], | ||
| ["Render Performance", color.bold(formatTime(avgBannerVisibility))], | ||
| ["Network Overhead", color.bold(formatBytes(avgNetworkImpact * KILOBYTE))], | ||
| ["Main Thread Impact", color.bold(formatTime(avgTBT))], | ||
| ["Layout Stability", color.bold(layoutStability)], | ||
| ["User Disruption", color.bold(`${avgViewportCoverage.toFixed(1)}%`)] | ||
| ); | ||
|
|
||
| console.log(summaryTable.toString()); | ||
|
|
||
| // ━━━ Network Chart (Waterfall) ━━━ | ||
| console.log(`\n${color.bold("🌐 Network Chart")}`); | ||
|
|
||
| // Get first iteration's resources for waterfall | ||
| const firstResult = results[0]; | ||
| if (firstResult?.resources) { | ||
| const allResources = [ | ||
| ...firstResult.resources.scripts.map((r) => ({ ...r, type: "script" })), | ||
| ...firstResult.resources.styles.map((r) => ({ ...r, type: "style" })), | ||
| ...firstResult.resources.images.map((r) => ({ ...r, type: "image" })), | ||
| ...firstResult.resources.fonts.map((r) => ({ ...r, type: "font" })), | ||
| ...firstResult.resources.other.map((r) => ({ ...r, type: "other" })), | ||
| ].sort((a, b) => a.startTime - b.startTime); | ||
|
|
||
| // Take top 10 resources for waterfall | ||
| const topResources = allResources.slice(0, 10); | ||
|
|
||
| if (topResources.length > 0) { | ||
| const maxEndTime = Math.max( | ||
| ...topResources.map((r) => r.startTime + r.duration) | ||
| ); | ||
| const chartWidth = 60; // Width of the waterfall bars | ||
|
|
||
| const waterfallTable = new Table({ | ||
| chars: { mid: "", "left-mid": "", "mid-mid": "", "right-mid": "" }, | ||
| colWidths: [COL_WIDTH_NAME, chartWidth + COL_WIDTH_CHART_PADDING], | ||
| style: { "padding-left": 1, "padding-right": 1, border: ["grey"] }, | ||
| wordWrap: true, | ||
| }); | ||
|
|
||
| waterfallTable.push([ | ||
| color.dim("Resource"), | ||
| color.dim( | ||
| "Timeline (0ms ───────────────────────────► " + | ||
| formatTime(maxEndTime) + | ||
| ")" | ||
| ), | ||
| ]); | ||
|
|
||
| for (const resource of topResources) { | ||
| const fileName = resource.name.split("/").pop() || resource.name; | ||
| const shortName = | ||
| fileName.length > MAX_FILENAME_LENGTH | ||
| ? `${fileName.substring(0, TRUNCATED_FILENAME_LENGTH)}...` | ||
| : fileName; | ||
|
|
||
| const startPos = Math.floor( | ||
| (resource.startTime / maxEndTime) * chartWidth | ||
| ); | ||
| const barLength = Math.max( | ||
| 1, | ||
| Math.floor((resource.duration / maxEndTime) * chartWidth) | ||
| ); | ||
|
|
||
| const emptyBefore = " ".repeat(startPos); | ||
| const bar = "█".repeat(barLength); | ||
| const durationLabel = | ||
| resource.duration > maxEndTime * MIN_DURATION_THRESHOLD | ||
| ? formatTime(resource.duration) | ||
| : ""; | ||
|
|
||
| let barColor = color.blue; | ||
| if (resource.isThirdParty) { | ||
| barColor = color.yellow; | ||
| } | ||
| if (resource.isCookieService) { | ||
| barColor = color.red; | ||
| } | ||
|
|
||
| waterfallTable.push([ | ||
| color.dim(shortName), | ||
| `${emptyBefore + barColor(bar)} ${color.dim(durationLabel)}`, | ||
| ]); | ||
| } | ||
|
|
||
| console.log(waterfallTable.toString()); | ||
| } | ||
| } | ||
|
|
||
| // ━━━ Resource Details ━━━ | ||
| console.log(`\n${color.bold("📋 Resource Details")}`); | ||
|
|
||
| // Aggregate resource data across all results | ||
| const aggregatedResources: Array<{ | ||
| name: string; | ||
| type: string; | ||
| source: string; | ||
| size: number; | ||
| duration: number; | ||
| tags: string[]; | ||
| }> = []; | ||
|
|
||
| // Use first result for resource list (assuming resources are consistent) | ||
| const sampleResult = results[0]; | ||
| if (sampleResult?.resources) { | ||
| const allSampleResources = [ | ||
| ...sampleResult.resources.scripts.map((r) => ({ | ||
| ...r, | ||
| type: "JavaScript", | ||
| })), | ||
| ...sampleResult.resources.styles.map((r) => ({ ...r, type: "CSS" })), | ||
| ...sampleResult.resources.images.map((r) => ({ ...r, type: "Image" })), | ||
| ...sampleResult.resources.fonts.map((r) => ({ ...r, type: "Font" })), | ||
| ...sampleResult.resources.other.map((r) => ({ ...r, type: "Other" })), | ||
| ]; | ||
|
|
||
| // Calculate averages for each resource | ||
| for (const sampleResource of allSampleResources) { | ||
| const resourceName = sampleResource.name; | ||
|
|
||
| // Find this resource in all results and average the values | ||
| const avgSize = | ||
| results.reduce((sum, result) => { | ||
| const allResources = [ | ||
| ...result.resources.scripts, | ||
| ...result.resources.styles, | ||
| ...result.resources.images, | ||
| ...result.resources.fonts, | ||
| ...result.resources.other, | ||
| ]; | ||
| const found = allResources.find((r) => r.name === resourceName); | ||
| return sum + (found ? found.size : 0); | ||
| }, 0) / results.length; | ||
|
|
||
| const avgDuration = | ||
| results.reduce((sum, result) => { | ||
| const allResources = [ | ||
| ...result.resources.scripts, | ||
| ...result.resources.styles, | ||
| ...result.resources.images, | ||
| ...result.resources.fonts, | ||
| ...result.resources.other, | ||
| ]; | ||
| const found = allResources.find((r) => r.name === resourceName); | ||
| return sum + (found ? found.duration : 0); | ||
| }, 0) / results.length; | ||
|
|
||
| let source = "Bundled"; | ||
| if (sampleResource.isThirdParty) { | ||
| source = sampleResource.isCookieService | ||
| ? "Cookie Service" | ||
| : "Third-Party"; | ||
| } | ||
|
|
||
| const tags: string[] = []; | ||
| if (!sampleResource.isThirdParty) { | ||
| tags.push("bundled"); | ||
| } | ||
| if (sampleResource.isThirdParty) { | ||
| tags.push("third-party"); | ||
| } | ||
| if (sampleResource.isCookieService) { | ||
| tags.push("cookie-service"); | ||
| } | ||
| if ("isDynamic" in sampleResource && sampleResource.isDynamic) { | ||
| tags.push("dynamic"); | ||
| } | ||
|
|
||
| // Add core/other categorization for bundled scripts | ||
| if ( | ||
| !sampleResource.isThirdParty && | ||
| sampleResource.type === "JavaScript" | ||
| ) { | ||
| tags.push("core"); | ||
| } | ||
|
|
||
| aggregatedResources.push({ | ||
| name: resourceName, | ||
| type: sampleResource.type, | ||
| source, | ||
| size: avgSize, | ||
| duration: avgDuration, | ||
| tags, | ||
| }); | ||
| } | ||
| } | ||
|
|
||
| // Sort by size (descending) and take top 10 | ||
| const topResources = aggregatedResources | ||
| .sort((a, b) => b.size - a.size) | ||
| .slice(0, 10); | ||
|
|
||
| if (topResources.length > 0) { | ||
| const detailsTable = new Table({ | ||
| head: ["Resource Name", "Type", "Source", "Size", "Duration", "Tags"], | ||
| colWidths: [ | ||
| COL_WIDTH_NAME, | ||
| COL_WIDTH_TYPE, | ||
| COL_WIDTH_SOURCE, | ||
| COL_WIDTH_SIZE, | ||
| COL_WIDTH_DURATION, | ||
| COL_WIDTH_TAGS, | ||
| ], | ||
| style: { head: ["cyan"], border: ["grey"] }, | ||
| wordWrap: true, | ||
| }); | ||
|
|
||
| for (const resource of topResources) { | ||
| const fileName = resource.name.split("/").pop() || resource.name; | ||
| const shortName = | ||
| fileName.length > MAX_FILENAME_LENGTH | ||
| ? `${fileName.substring(0, TRUNCATED_FILENAME_LENGTH)}...` | ||
| : fileName; | ||
|
|
||
| let sourceColor = color.green; | ||
| if (resource.source === "Third-Party") { | ||
| sourceColor = color.yellow; | ||
| } | ||
| if (resource.source === "Cookie Service") { | ||
| sourceColor = color.red; | ||
| } | ||
|
|
||
| detailsTable.push([ | ||
| shortName, | ||
| resource.type, | ||
| sourceColor(resource.source), | ||
| formatBytes(resource.size * KILOBYTE), | ||
| color.blue(formatTime(resource.duration)), | ||
| resource.tags.join(", "), | ||
| ]); | ||
| } | ||
|
|
||
| console.log(detailsTable.toString()); | ||
| } | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider extracting display sections into separate functions.
The printDetailedResults function spans 467 lines and handles multiple distinct display sections (score, insights, banner impact, web vitals, resources, etc.). This violates the Single Responsibility Principle and makes the function difficult to test and maintain.
Consider extracting each display section into its own function (e.g., printScoreSection, printWebVitalsSection, printResourceBreakdown, etc.).
🤖 Prompt for AI Agents
In packages/cookiebench-cli/src/commands/results.ts around lines 421 to 888, the
printDetailedResults function is too large and mixes multiple display
responsibilities; extract each major display section into its own function
(e.g., printScoreSection(appName, scores), printInsightsSection(scores),
printBannerImpactSection(avgBannerVisibility, avgViewportCoverage,
avgNetworkImpact, isBundled, bannerDelta), printWebVitalsSection(avgFCP, avgLCP,
avgTTI, avgCLS), printResourceBreakdownSection(totalSize, jsSize, cssSize,
imageSize, fontSize, otherSize, jsFiles, cssFiles, imageFiles, fontFiles,
otherFiles), printSummarySection(...), printWaterfallSection(firstResult), and
printResourceDetailsSection(aggregatedResources)). Move all console/table
rendering logic into these helpers, keep metric calculations in
printDetailedResults (or extract a computeMetrics(results, baseline) helper that
returns the averages/flags used by the printers), pass only the minimal data
each printer needs, and keep existing output formatting intact; update
imports/exports as needed and add small unit tests for the new functions to
verify they render the same strings/tables as before.
| const avgNetworkImpact = | ||
| results.reduce((a, b) => a + b.size.thirdParty, 0) / results.length; | ||
| const _bannerDetected = results.some((r) => r.timing.cookieBanner.detected); | ||
| const isBundled = results[0]?.size.thirdParty === 0; |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Verify bundled status across all results.
The bundled status is determined from only the first result. If results are inconsistent across iterations, this could give incorrect information.
Consider checking all results for consistency or using a threshold:
-const isBundled = results[0]?.size.thirdParty === 0;
+const isBundled = results.every((r) => r.size.thirdParty === 0);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| const isBundled = results[0]?.size.thirdParty === 0; | |
| const isBundled = results.every((r) => r.size.thirdParty === 0); |
🤖 Prompt for AI Agents
In packages/cookiebench-cli/src/commands/results.ts around line 469, the bundled
status is computed only from results[0], which can be wrong if iterations
differ; update the logic to examine all results instead (e.g., set isBundled =
results.every(r => r?.size.thirdParty === 0) or use a threshold/majority like
count of results with thirdParty===0 divided by results.length) and emit a
warning or note when results are inconsistent so callers know a consensus was
not reached.
| appResults.reduce((a, b) => a + (b.timing.timeToFirstByte || 0), 0) / | ||
| appResults.length, | ||
| interactionToNextPaint: | ||
| appResults[0]?.timing.interactionToNextPaint || null, |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider averaging interactionToNextPaint across all results.
The interactionToNextPaint metric uses only the first result instead of averaging across all iterations. While INP may not be present in all results (hence the || null), if multiple results have values, they should be averaged for consistency with other metrics.
Consider averaging when values are available:
- interactionToNextPaint:
- appResults[0]?.timing.interactionToNextPaint || null,
+ interactionToNextPaint: (() => {
+ const values = appResults
+ .map((r) => r.timing.interactionToNextPaint)
+ .filter((v): v is number => v !== null && v !== undefined);
+ return values.length > 0
+ ? values.reduce((a, b) => a + b, 0) / values.length
+ : null;
+ })(),📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| appResults[0]?.timing.interactionToNextPaint || null, | |
| interactionToNextPaint: (() => { | |
| const values = appResults | |
| .map((r) => r.timing.interactionToNextPaint) | |
| .filter((v): v is number => v !== null && v !== undefined); | |
| return values.length > 0 | |
| ? values.reduce((a, b) => a + b, 0) / values.length | |
| : null; | |
| })(), |
🤖 Prompt for AI Agents
In packages/cookiebench-cli/src/commands/results.ts around line 1021, the code
currently selects interactionToNextPaint from only the first result; change this
to compute the average across all results that have a defined non-null
interactionToNextPaint value and return null if none exist. Iterate over
appResults, collect numeric interactionToNextPaint values, sum and divide by the
count to produce the mean (preserving numeric type), and replace the
single-index access with that averaged value so it matches how other metrics are
aggregated.
| thirdPartySize: | ||
| appResults.reduce((a, b) => a + b.size.thirdParty, 0) / | ||
| appResults.length, | ||
| thirdPartyDomains: 5, |
There was a problem hiding this comment.
Replace hardcoded thirdPartyDomains with calculated value.
The thirdPartyDomains metric is hardcoded to 5, which produces incorrect scoring for network impact. This value should be calculated from the actual benchmark data by counting unique third-party domains.
Calculate the actual count of unique third-party domains:
- thirdPartyDomains: 5,
+ thirdPartyDomains: new Set(
+ appResults.flatMap((result) =>
+ [
+ ...result.resources.scripts,
+ ...result.resources.styles,
+ ...result.resources.images,
+ ...result.resources.fonts,
+ ...result.resources.other,
+ ]
+ .filter((r) => r.isThirdParty)
+ .map((r) => {
+ try {
+ return new URL(r.name).hostname;
+ } catch {
+ return r.name.split('/')[2] || r.name;
+ }
+ })
+ )
+ ).size,📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| thirdPartyDomains: 5, | |
| thirdPartyDomains: new Set( | |
| appResults.flatMap((result) => | |
| [ | |
| ...result.resources.scripts, | |
| ...result.resources.styles, | |
| ...result.resources.images, | |
| ...result.resources.fonts, | |
| ...result.resources.other, | |
| ] | |
| .filter((r) => r.isThirdParty) | |
| .map((r) => { | |
| try { | |
| return new URL(r.name).hostname; | |
| } catch { | |
| return r.name.split('/')[2] || r.name; | |
| } | |
| }) | |
| ) | |
| ).size, |
| if (trimmed.length === 0) { | ||
| return sorted[Math.floor(sorted.length / 2)]; | ||
| } |
There was a problem hiding this comment.
Fix trimmed-mean fallback for even-length datasets.
When trimming drops every element (e.g., trimPercent ≥ 50 on an even-sized sample), sorted[Math.floor(sorted.length / 2)] returns only the upper middle value, inflating the mean instead of falling back to the true median. Reuse your existing getMedian helper here.
Apply this diff:
- if (trimmed.length === 0) {
- return sorted[Math.floor(sorted.length / 2)];
- }
+ if (trimmed.length === 0) {
+ return getMedian(sorted);
+ }📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if (trimmed.length === 0) { | |
| return sorted[Math.floor(sorted.length / 2)]; | |
| } | |
| if (trimmed.length === 0) { | |
| return getMedian(sorted); | |
| } |
🤖 Prompt for AI Agents
In packages/runner/src/statistics.ts around lines 102 to 104, the trimmed-mean
fallback uses sorted[Math.floor(sorted.length / 2)] which returns the upper
middle for even-length arrays and inflates the result; replace that direct index
access with a call to the existing getMedian(sorted) helper so the fallback
returns the true median for both even and odd lengths.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 11
♻️ Duplicate comments (7)
packages/shared/package.json (1)
21-21: Empty dependencies object can be omitted.The empty
"dependencies"object is unnecessary and should be removed to keep the manifest clean.- "dependencies": {}, "devDependencies": {packages/shared/tsconfig.json (1)
1-14: Adddeclaration: trueto emit TypeScript declaration files.As a shared utilities package consumed by other packages in the monorepo (benchmark, runner, cookiebench-cli), emitting
.d.tsfiles is essential for type safety and IDE support.Apply this diff:
{ "compilerOptions": { "target": "ES2020", "module": "ESNext", "moduleResolution": "bundler", "esModuleInterop": true, "strict": true, "skipLibCheck": true, "outDir": "dist", - "rootDir": "src" + "rootDir": "src", + "declaration": true, + "declarationMap": true }, "include": ["src/**/*"], "exclude": ["node_modules", "dist"] }packages/shared/src/constants.ts (2)
6-7: Consolidate duplicate constants with identical values.
BYTES_TO_KBandKILOBYTEboth equal 1024. Having two names for the same constant creates confusion about which to use.Consider consolidating:
// Size constants (bytes to kilobytes) -export const BYTES_TO_KB = 1024; -export const KILOBYTE = 1024; +export const KILOBYTE = 1024; +export const BYTES_TO_KB = KILOBYTE; // Alias for conversion contextsOr pick one name and update all usages across the codebase.
10-11: Clarify or consolidate percentage constants.
PERCENTAGE_MULTIPLIERandPERCENTAGE_DIVISORboth equal 100. If they represent conceptually different operations (e.g., converting to/from percentage), document why they share the same value. Otherwise, use a single constant.Either consolidate to a single name:
// Percentage constants -export const PERCENTAGE_MULTIPLIER = 100; -export const PERCENTAGE_DIVISOR = 100; +export const PERCENTAGE_BASE = 100;Or add a comment explaining the semantic distinction between multiplication and division contexts.
packages/cookiebench-cli/src/commands/scores.ts (1)
211-318: Guard against empty result sets before averaging.If
appResultsis empty (e.g., a failed run wrote an empty results array), every average computed on lines 231-317 divides by0, producingNaNscores. This issue was previously flagged and remains unaddressed.Bail out early with a clear error before calling
calculateScores:const appResults = result.results; + if (!Array.isArray(appResults) || appResults.length === 0) { + logger.error(`No iterations recorded for ${appName}; cannot compute scores.`); + return; + } + const config = await loadConfigForApp(logger, appName);packages/runner/src/performance-aggregator.ts (2)
133-137: Redundant fallback still present despite prior review.The division on line 135 uses
(totalBlockingTime || 1)as the denominator, but this fallback is unnecessary since line 134's ternary already ensures the division only occurs whentotalBlockingTime > 0. A past review marked this as addressed in commit c79d556, but the redundant fallback remains in the code.Apply this diff to remove the redundant fallback:
const percentageFromCookies = totalBlockingTime > 0 - ? (cookieBannerEstimate / (totalBlockingTime || 1)) * + ? (cookieBannerEstimate / totalBlockingTime) * PERCENTAGE_MULTIPLIER : 0;
179-179: Inconsistent nullish coalescing for navigationTiming.Line 179 uses logical OR (
||) for thenavigationTimingfallback, but a past review requested changing this to nullish coalescing (??) to avoid converting falsy but valid values. While lines 176-178 correctly adopted??, this line was missed.Apply this diff for consistency:
- navigationTiming: perfumeMetrics?.navigationTiming || { + navigationTiming: perfumeMetrics?.navigationTiming ?? { timeToFirstByte: 0, domInteractive: 0, domContentLoadedEventStart: 0, domContentLoadedEventEnd: 0, domComplete: 0, loadEventStart: 0, loadEventEnd: 0, },
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (13)
METHODOLOGY.md(1 hunks)packages/cookiebench-cli/src/commands/benchmark.ts(1 hunks)packages/cookiebench-cli/src/commands/scores.ts(1 hunks)packages/cookiebench-cli/src/utils/index.ts(1 hunks)packages/runner/package.json(1 hunks)packages/runner/src/performance-aggregator.ts(1 hunks)packages/shared/package.json(1 hunks)packages/shared/rslib.config.ts(1 hunks)packages/shared/src/constants.ts(1 hunks)packages/shared/src/utils/config.ts(1 hunks)packages/shared/src/utils/package-manager.ts(1 hunks)packages/shared/src/utils/time.ts(1 hunks)packages/shared/tsconfig.json(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (8)
packages/shared/src/utils/time.ts (4)
packages/cookiebench-cli/src/utils/index.ts (2)
formatTime(9-9)ONE_SECOND(13-13)packages/runner/src/index.ts (1)
formatTime(31-31)packages/shared/src/index.ts (2)
formatTime(22-22)ONE_SECOND(8-8)packages/shared/src/constants.ts (1)
ONE_SECOND(2-2)
packages/cookiebench-cli/src/commands/scores.ts (5)
packages/cookiebench-cli/src/commands/results.ts (2)
BenchmarkOutput(179-213)RawBenchmarkDetail(37-177)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/benchmark/src/types.ts (1)
Config(13-56)packages/shared/src/constants.ts (2)
HALF_SECOND(3-3)PERCENTAGE_DIVISOR(11-11)packages/cookiebench-cli/src/utils/scoring.ts (2)
printScores(1383-1422)calculateScores(1010-1379)
packages/shared/src/constants.ts (2)
packages/cookiebench-cli/src/utils/index.ts (5)
ONE_SECOND(13-13)HALF_SECOND(11-11)KILOBYTE(12-12)PERCENTAGE_MULTIPLIER(15-15)PERCENTAGE_DIVISOR(14-14)packages/shared/src/index.ts (7)
ONE_SECOND(8-8)HALF_SECOND(6-6)BYTES_TO_KB(5-5)KILOBYTE(7-7)PERCENTAGE_MULTIPLIER(10-10)PERCENTAGE_DIVISOR(9-9)TTI_BUFFER_MS(11-11)
packages/cookiebench-cli/src/utils/index.ts (3)
packages/shared/src/utils/config.ts (1)
readConfig(14-26)packages/runner/src/utils.ts (1)
readConfig(6-16)packages/benchmark/src/types.ts (1)
Config(13-56)
packages/shared/src/utils/config.ts (1)
packages/cookiebench-cli/src/utils/index.ts (1)
readConfig(21-23)
packages/shared/src/utils/package-manager.ts (3)
packages/cookiebench-cli/src/utils/index.ts (1)
getPackageManager(10-10)packages/runner/src/index.ts (1)
getPackageManager(31-31)packages/shared/src/index.ts (1)
getPackageManager(21-21)
packages/cookiebench-cli/src/commands/benchmark.ts (11)
packages/runner/src/index.ts (6)
BenchmarkResult(11-11)readConfig(31-31)ServerInfo(21-21)buildAndServeNextApp(7-7)BenchmarkRunner(3-3)cleanupServer(7-7)packages/runner/src/types.ts (2)
BenchmarkResult(158-271)ServerInfo(18-21)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(162-162)CliLogger(9-9)packages/cookiebench-cli/src/utils/index.ts (3)
PERCENTAGE_DIVISOR(14-14)readConfig(21-23)HALF_SECOND(11-11)packages/shared/src/constants.ts (2)
PERCENTAGE_DIVISOR(11-11)HALF_SECOND(3-3)packages/cookiebench-cli/src/utils/constants.ts (3)
DEFAULT_DOM_SIZE(3-3)DEFAULT_ITERATIONS(2-2)SEPARATOR_WIDTH(5-5)packages/runner/src/benchmark-runner.ts (1)
BenchmarkRunner(30-474)packages/shared/src/utils/config.ts (1)
readConfig(14-26)packages/runner/src/server.ts (2)
buildAndServeNextApp(6-71)cleanupServer(73-77)packages/cookiebench-cli/src/utils/scoring.ts (2)
calculateScores(1010-1379)printScores(1383-1422)packages/cookiebench-cli/src/commands/results.ts (1)
resultsCommand(890-1120)
packages/runner/src/performance-aggregator.ts (4)
packages/benchmark/src/types.ts (7)
CoreWebVitals(216-227)CookieBannerData(94-103)CookieBannerMetrics(79-92)NetworkRequest(106-113)NetworkMetrics(115-118)ResourceTimingData(128-213)PerfumeMetrics(230-261)packages/runner/src/types.ts (9)
CoreWebVitals(10-10)CookieBannerData(8-8)CookieBannerMetrics(9-9)NetworkRequest(12-12)NetworkMetrics(11-11)ResourceTimingData(14-14)PerfumeMetrics(13-13)BenchmarkDetails(24-156)BenchmarkResult(158-271)packages/shared/src/constants.ts (2)
TTI_BUFFER_MS(14-14)PERCENTAGE_MULTIPLIER(10-10)packages/runner/src/statistics.ts (4)
isStable(132-138)calculateCoefficientOfVariation(112-127)calculateTrimmedMean(84-107)calculateStatistics(11-52)
🪛 LanguageTool
METHODOLOGY.md
[grammar] ~79-~79: Ensure spelling is correct
Context: ...aintains speed - Fast 3G (1.6 Mbps, 562ms latency): More realistic, significantly...
(QB_NEW_EN_ORTHOGRAPHY_ERROR_IDS_1)
🔇 Additional comments (27)
packages/shared/package.json (1)
1-29: Package manifest structure looks solid.The exports configuration, entry points, and build tooling are well-configured for a TypeScript library in this monorepo. Once you remove the empty dependencies object (line 21), this is ready to go.
packages/shared/rslib.config.ts (1)
1-19: LGTM! Standard rslib configuration.The configuration follows best practices for a shared package in a Node.js monorepo, with appropriate settings for ESM output, TypeScript declarations, and ES2021 syntax.
packages/runner/package.json (4)
1-13: Solid package structure with dual exports for maximum compatibility.The use of both the modern
exportsfield (with types) and legacymain/module/typesfields is best practice for universal packages. This ensures ESM and CJS consumers, as well as TypeScript tooling, can properly resolve the package.
22-22: Verify the strict version pinning on @c15t/logger.The
@c15t/loggerdependency is pinned to an exact version (1.0.0) without flexibility modifiers (^or~). Confirm whether this strict pinning is intentional for stability or if it should allow minor/patch updates for security fixes and improvements.
29-32: Verify dev dependency versions align with workspace conventions.Confirm that
@rslib/core,typescript, and other dev tools are version-consistent with those used in other workspace packages to avoid tooling conflicts.
25-26: I'll search for the latest versions of both packages to verify stability and compatibility.I need to search for the correct package name to verify the version:
Let me search more specifically for the npm package:
The package
playwright-performance-metrics@^1.2.2does not appear to exist on npm and will fail installation.@playwright/test is currently at version 1.56.1, which is the latest stable release. However, playwright-performance-metrics is documented on GitHub but doesn't appear as a published npm package. The npm registry search returns no results for this exact package name.
Verify that:
- The package name is correct (similar packages on npm include
playwright-performanceat v2.0.5,playwright-lighthouse, and others)- If
playwright-performance-metricsis intentional, check whether it's a private package or needs to be installed from the GitHub repository directly- The version constraint
^1.2.2aligns with what's actually availablepackages/shared/src/utils/config.ts (1)
20-20: Verify JSON structure matches the expected type.The type cast
as Tprovides no runtime validation that the parsed JSON matches the expected config schema. Invalid configs will only cause errors downstream.Consider adding runtime validation using a schema validator (e.g., Zod, io-ts) or document that consumers must validate configs themselves. For critical config properties, at least verify their presence:
const parsed = JSON.parse(configContent); if (!parsed || typeof parsed !== 'object') { throw new Error('Config must be a valid object'); } return parsed as T;packages/cookiebench-cli/src/utils/index.ts (1)
1-23: Excellent refactoring to eliminate duplication.The delegation to
@consentio/sharedeliminates the code duplication previously flagged in past reviews. This centralized approach ensures consistent behavior across packages and simplifies maintenance.METHODOLOGY.md (1)
1-289: LGTM! Comprehensive and well-structured methodology documentation.The methodology document is thorough and clearly explains:
- The distinction between Banner Render Time and Banner Visibility Time
- Opacity threshold (0.5) for user-perceived visibility
- Measurement baseline from navigationStart
- Network conditions and their impact
- Primary and secondary metrics with scoring methodology
- Reproducibility requirements and limitations
The documentation aligns with industry standards (Core Web Vitals, Lighthouse, WebPageTest, W3C) and maintains transparency about measurement approaches and limitations.
packages/cookiebench-cli/src/commands/benchmark.ts (5)
35-40: LGTM! Empty array handling implemented correctly.The function now properly guards against division by zero by returning 0 for empty arrays, preventing NaN propagation through metric calculations.
214-226: LGTM! DOM size now uses actual data with safe fallback.The function correctly extracts DOM size from benchmark results using optional chaining and provides a sensible default when data is unavailable.
248-383: LGTM! Robust implementation with proper error handling and cleanup.The function demonstrates good practices:
- Validates results before processing (lines 301-304)
- Supports both remote and local benchmarking
- Ensures server cleanup in finally block
- Returns boolean status instead of calling process.exit()
- Creates necessary directories with error handling
- Enables trace saving for debugging
388-572: LGTM! Well-structured interactive command with excellent UX.The main command function demonstrates strong implementation:
- Proper error handling without process.exit() (lines 396, 415)
- Clear user prompts and progress indicators
- Configurable iterations with per-benchmark defaults
- Sequential execution with failure tracking
- Optional results panel via dynamic import
- Clean cancellation handling throughout
The interactive flow provides a professional CLI experience.
101-121: The review comment is incorrect—resource.nameis the correct property.The codebase defines resource objects with a
nameproperty containing the URL string (from the PerformanceResourceTiming API), not aurlproperty. Line 113 of benchmark.ts correctly usesresource.nameto extract the hostname. Nourlproperty exists on resource objects anywhere in the codebase.Likely an incorrect or invalid review comment.
packages/cookiebench-cli/src/commands/scores.ts (1)
218-226: No action needed—theJSON.stringifycalls are correct.The
AppDatatype definition confirms all relevant fields (company,techStack,source,tags) must be strings or null, not objects. TheJSON.stringifycalls are necessary and properly convert object values to match the expected type. No changes required.packages/runner/src/performance-aggregator.ts (12)
1-19: LGTM: Clean imports with proper constant reuse.The imports correctly leverage shared constants (line 12), addressing the code duplication concern from previous reviews.
21-24: LGTM: Well-defined local configuration constants.
25-34: LGTM: Comprehensive parameter type definition.
36-41: LGTM: Clean dependency injection.
42-56: LGTM: Sound TTI calculation logic.The use of
Math.maxwith appropriate fallbacks and buffer addition correctly implements the Time to Interactive metric.
58-96: LGTM: Well-documented UX-focused visibility metrics.The visibility time fallback chain correctly prioritizes user-perceived timing over technical render time, with excellent documentation explaining the rationale.
149-175: LGTM: Comprehensive metric aggregation.The
aggregateMetricsmethod effectively combines all performance data sources into a unifiedBenchmarkDetailsstructure. The visibility time handling in lines 207-210 correctly implements the user-perceived timing approach.Also applies to: 189-225
227-241: LGTM: Clean network impact calculation.
243-367: LGTM: Robust statistical aggregation with outlier handling.The use of trimmed mean (10% trim) and coefficient of variation checks provides resilient averaging. The variability warnings (lines 287-301) offer valuable observability into measurement stability.
369-389: LGTM: Clean statistical summary extraction.
391-423: LGTM: Comprehensive structured logging.The debug output captures all essential performance metrics and configuration details for effective troubleshooting.
425-463: LGTM: Informative statistical summary output.The method provides clear, actionable insights into measurement stability with both quantitative statistics and qualitative indicators.
| } catch { | ||
| // Directory doesn't exist or can't be read | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider logging suppressed errors in debug mode.
The empty catch block silently swallows all errors, which may hide actionable issues like permission problems or I/O failures that differ from a non-existent directory.
Apply this diff to log errors in debug mode:
- } catch {
- // Directory doesn't exist or can't be read
+ } catch (error) {
+ // Directory doesn't exist or can't be read - this is expected in some cases
+ // Optionally log at debug level if a logger is available
}Note: Since logger is not available in this helper, you could either pass it as a parameter or accept that some errors remain hidden.
Committable suggestion skipped: line range outside the PR's diff.
🤖 Prompt for AI Agents
In packages/cookiebench-cli/src/commands/scores.ts around lines 37 to 39, the
empty catch block is swallowing all errors; change it to catch the error (catch
(err)) and, in debug mode, log the error details instead of ignoring them—either
accept a logger parameter and call logger.debug("reading scores directory
failed", err) or, if adding logger is undesirable, call console.debug with the
error; keep the original behavior for non-debug runs (silently ignore
non-actionable cases like missing directory).
| export function readConfig<T extends BaseConfig = BaseConfig>( | ||
| configPath?: string | ||
| ): T | null { | ||
| try { | ||
| const path = configPath || join(process.cwd(), "config.json"); | ||
| const configContent = readFileSync(path, "utf-8"); | ||
| return JSON.parse(configContent) as T; | ||
| } catch (error) { | ||
| // biome-ignore lint/suspicious/noConsole: console error is needed for debugging | ||
| console.error("Failed to read config.json:", error); | ||
| return null; | ||
| } | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Consider using async file I/O to avoid blocking the event loop.
readFileSync blocks the Node.js event loop during config reads. For large config files or startup performance, an async alternative would be preferable.
Consider this async alternative:
import { readFile } from "node:fs/promises";
export async function readConfig<T extends BaseConfig = BaseConfig>(
configPath?: string
): Promise<T | null> {
try {
const path = configPath || join(process.cwd(), "config.json");
const configContent = await readFile(path, "utf-8");
return JSON.parse(configContent) as T;
} catch (error) {
console.error("Failed to read config.json:", error);
return null;
}
}If synchronous behavior is required for CLI startup simplicity, document this decision in a comment.
🤖 Prompt for AI Agents
In packages/shared/src/utils/config.ts around lines 14 to 26, the function
currently uses readFileSync which blocks the event loop; change it to use the
async fs/promises API (import readFile from "node:fs/promises"), update the
function signature to async and return Promise<T | null>, replace readFileSync
call with await readFile(path, "utf-8"), and keep the same try/catch and error
logging; also update all call sites to await the new async function (or if
synchronous startup behavior is required, add a clear comment explaining why
readFileSync is intentionally used and keep a synchronous wrapper only for those
specific CLI paths).
…benchmarks. Enhance Iubenda layout script comment for clarity. Refactor bundle strategy and cookie banner collector logic for improved accuracy in performance metrics. Update PerfumeCollector to utilize navigation timing API and handle network information more robustly. Introduce first-party resource detection in ResourceTimingCollector and improve language detection logic.
…ror handling. Update performance metrics to handle optional properties more robustly, ensuring consistent data retrieval and preventing potential runtime errors.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 8
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (7)
benchmarks/with-enzuzo/package.json(2 hunks)benchmarks/with-iubenda/app/layout.tsx(2 hunks)benchmarks/with-osano/package.json(2 hunks)packages/benchmark/src/bundle-strategy.ts(1 hunks)packages/benchmark/src/cookie-banner-collector.ts(1 hunks)packages/benchmark/src/perfume-collector.ts(1 hunks)packages/benchmark/src/resource-timing-collector.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (4)
packages/benchmark/src/resource-timing-collector.ts (2)
packages/benchmark/src/types.ts (1)
ResourceTimingData(128-213)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(17-74)
packages/benchmark/src/bundle-strategy.ts (2)
packages/benchmark/src/types.ts (2)
Config(13-56)BundleStrategy(121-125)packages/benchmark/src/constants.ts (1)
BUNDLE_TYPES(76-81)
packages/benchmark/src/perfume-collector.ts (1)
packages/benchmark/src/types.ts (2)
WindowWithPerfumeMetrics(263-279)PerfumeMetrics(230-261)
packages/benchmark/src/cookie-banner-collector.ts (3)
packages/benchmark/src/types.ts (5)
Config(13-56)CookieBannerMetrics(79-92)WindowWithCookieMetrics(65-77)LayoutShiftEntry(59-62)CookieBannerData(94-103)packages/benchmark/src/bundle-strategy.ts (1)
determineBundleStrategy(4-24)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(17-74)
🪛 ast-grep (0.39.6)
benchmarks/with-iubenda/app/layout.tsx
[warning] 18-18: Usage of dangerouslySetInnerHTML detected. This bypasses React's built-in XSS protection. Always sanitize HTML content using libraries like DOMPurify before injecting it into the DOM to prevent XSS attacks.
Context: dangerouslySetInnerHTML
Note: [CWE-79] Improper Neutralization of Input During Web Page Generation [REFERENCES]
- https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml
- https://cwe.mitre.org/data/definitions/79.html
(react-unsafe-html-injection)
🔇 Additional comments (16)
benchmarks/with-iubenda/app/layout.tsx (1)
18-85: Comment improves suppression justification and addresses previous feedback.The biome-ignore comment on Line 18 has been properly updated with specific, accurate justification: the Iubenda configuration is hardcoded and contains no user input, making the use of
dangerouslySetInnerHTMLsafe. This directly addresses the previous review's concern about grammar and justification.The script tag attribute reordering on Line 84 is a minor cosmetic change with no functional impact.
benchmarks/with-osano/package.json (1)
6-6: ✅ Previous issues resolved: port conflict, package migration, and engines field.Mirrors the improvements in
with-enzuzo:
- Line 6: Port updated from 3006 → 3007 (resolves conflict with
with-onetrust).- Line 22: Package migrated from
@cookiebench/cli→cookiebench.- Lines 25–27:
enginesfield correctly named with Node constraint.Verify port 3007 does not conflict with other benchmarks (see script in
with-enzuzoreview for comprehensive port audit).Also applies to: 22-22, 25-27
benchmarks/with-enzuzo/package.json (1)
6-6: ✅ All previous issues resolved and port conflict verified.The changes correctly address earlier flagged issues:
- Line 6: Port updated from 3001 → 3002 (resolves conflict; 3002 is unique across benchmark packages).
- Line 22: Package migrated from
@cookiebench/cli→cookiebench.- Lines 25–27:
enginesfield correctly named (plural) with proper Node constraint.packages/benchmark/src/bundle-strategy.ts (1)
1-1: The IIFE typo fix is complete and verified across the codebase.The constant
BUNDLE_TYPES.IIFEis correctly spelled as"iife"in constants.ts (line 77), no instances of the misspelling"iffe"remain, and all benchmark configuration files consistently use the correct spelling.packages/benchmark/src/resource-timing-collector.ts (4)
1-11: LGTM! Clean dependency injection.The imports are appropriate and the constructor properly injects the logger dependency for use throughout the class.
27-39: Excellent fix! Third-party detection now accurate.The
isFirstPartyhelper properly addresses the previous review comment by using URL parsing and hostname comparison instead of substring matching. The fallback handling for relative URLs and malformed URLs is also correct.
173-180: Excellent fix! Language detection now accurate.The language detection properly addresses the previous review comment by reading the actual document language with appropriate fallbacks to
navigator.language,navigator.languages[0], and finally"en".
81-96: Verify placeholder timing values.Several timing fields are hardcoded to 0 (
loadStart,executeStart,executeEnd). Ensure these are intentional placeholders or confirm whether the Resource Timing API provides data to populate them (e.g.,responseStart,responseEndcould map to load times).packages/benchmark/src/cookie-banner-collector.ts (3)
1-23: LGTM: Clear documentation of opacity threshold.The imports are appropriate and the OPACITY_VISIBILITY_THRESHOLD constant is well-documented with a clear rationale for the 50% threshold to account for CSS animations.
37-70: LGTM: Proper initialization with bundle strategy detection.The method correctly determines the bundle strategy, logs it for debugging, and returns a properly initialized metrics object.
275-366: LGTM: Comprehensive metrics collection with proper fallback logic.The metrics collection correctly computes render time, visibility time with fallback, interactive time, layout shift impact, and viewport coverage. The viewport intersection calculation properly handles partial visibility.
Note: The bannerHydrationTime calculation at lines 316-319 has already been flagged in a previous review for missing the
metrics.detectedcheck.packages/benchmark/src/perfume-collector.ts (5)
10-15: LGTM!Clean constructor with proper dependency injection.
86-158: LGTM!Robust script loading with multiple fallback strategies for different monorepo layouts. The createRequire fallback provides good coverage, and the warning ensures visibility when all attempts fail.
163-176: LGTM!The metrics collection waits for Perfume.js to populate data and logs appropriately. The timeout constant was previously flagged as potentially configurable, but using a constant is acceptable for now.
177-195: LGTM! Previous deprecation issue resolved.Now correctly uses the modern Performance Timeline API (
PerformanceNavigationTiming) instead of the deprecatedperformance.timing. The null check properly handles cases where navigation entries aren't available.
198-226: LGTM! Previous experimental API concerns resolved.Excellent defensive programming: checks for
navigatoravailability, wraps in try/catch, includes vendor prefixes, and provides sensible defaults. The optional return type properly matches thePerfumeMetricsinterface.
| if ( | ||
| isVisible && | ||
| (bannerMetrics.bannerVisibleTime === 0 || | ||
| now < bannerMetrics.bannerVisibleTime) | ||
| ) { | ||
| bannerMetrics.bannerVisibleTime = now; | ||
| } |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Visibility time update condition may be unnecessary.
The condition now < bannerMetrics.bannerVisibleTime allows updating to an earlier time, but given the linear polling nature of detection, bannerVisibleTime should only be set once. The bannerVisibleTime === 0 check alone should be sufficient.
Consider simplifying:
-if (
- isVisible &&
- (bannerMetrics.bannerVisibleTime === 0 ||
- now < bannerMetrics.bannerVisibleTime)
-) {
+if (isVisible && bannerMetrics.bannerVisibleTime === 0) {
bannerMetrics.bannerVisibleTime = now;
}If the "earlier time" logic is intentionally defensive against edge cases, consider adding a comment explaining the scenario it protects against.
📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| if ( | |
| isVisible && | |
| (bannerMetrics.bannerVisibleTime === 0 || | |
| now < bannerMetrics.bannerVisibleTime) | |
| ) { | |
| bannerMetrics.bannerVisibleTime = now; | |
| } | |
| if (isVisible && bannerMetrics.bannerVisibleTime === 0) { | |
| bannerMetrics.bannerVisibleTime = now; | |
| } |
🤖 Prompt for AI Agents
In packages/benchmark/src/cookie-banner-collector.ts around lines 195 to 201,
the visibility-time update currently allows replacing bannerVisibleTime with an
earlier timestamp via the now < bannerMetrics.bannerVisibleTime clause; simplify
by only setting bannerMetrics.bannerVisibleTime when it is still 0 (i.e., if
(isVisible && bannerMetrics.bannerVisibleTime === 0)
bannerMetrics.bannerVisibleTime = now), or if you intend to keep the
earlier-time guard, add a concise comment explaining the specific edge case it
defends against so future readers understand why an earlier timestamp would be
preferred.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 4
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
📒 Files selected for processing (1)
packages/cookiebench-cli/src/commands/benchmark.ts(1 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
packages/cookiebench-cli/src/commands/benchmark.ts (8)
packages/runner/src/types.ts (2)
BenchmarkResult(158-271)ServerInfo(18-21)packages/cookiebench-cli/src/utils/index.ts (3)
PERCENTAGE_DIVISOR(14-14)readConfig(21-23)HALF_SECOND(11-11)packages/shared/src/constants.ts (2)
PERCENTAGE_DIVISOR(11-11)HALF_SECOND(3-3)packages/cookiebench-cli/src/utils/constants.ts (3)
DEFAULT_DOM_SIZE(3-3)DEFAULT_ITERATIONS(2-2)SEPARATOR_WIDTH(5-5)packages/runner/src/benchmark-runner.ts (2)
runSingleBenchmark(96-199)BenchmarkRunner(30-474)packages/shared/src/utils/config.ts (1)
readConfig(14-26)packages/runner/src/server.ts (2)
buildAndServeNextApp(6-71)cleanupServer(73-77)packages/cookiebench-cli/src/utils/scoring.ts (2)
calculateScores(1010-1379)printScores(1383-1422)
🔇 Additional comments (10)
packages/cookiebench-cli/src/commands/benchmark.ts (10)
1-30: LGTM - Clean import structure.All imports are properly organized and appear to be used throughout the file. The separation between node built-ins, third-party packages, and local utilities is clear.
35-40: LGTM - Proper empty array handling.The guard at line 36 correctly prevents NaN from division by zero when the array is empty.
234-246: LGTM - Robust directory scanning.The function properly filters hidden directories and handles errors gracefully by returning an empty array, which is appropriate for the benchmark discovery flow.
304-307: LGTM - Proper result validation.The validation ensures
result.detailsexists and contains at least one iteration before attempting metric calculations, preventing division by zero errors incalculateAverage.
372-377: LGTM - Proper resource cleanup.The
finallyblock ensures server cleanup only when a local server was started, preventing errors when using remote benchmarking.
391-402: LGTM - Proper error handling for direct benchmark execution.The function now throws an error instead of calling
process.exit(), allowing the CLI entry point to handle the exit code gracefully. This also makes the code more testable.
413-419: LGTM - Proper error handling for missing benchmarks.The function throws an error instead of calling
process.exit(), following the same pattern as the previous fix and allowing proper error propagation.
473-504: LGTM - Robust iteration input validation.The validation logic properly handles empty input (to use config defaults) and validates numeric input, with clear error messages. The parsing with
parseInt(value, 10)correctly specifies the radix.
517-547: LGTM - Resilient sequential execution.The benchmark execution loop properly:
- Tracks success/failure for each benchmark
- Continues execution even if individual benchmarks fail (good for batch runs)
- Provides clear progress indicators with
[i/total]format- Adds visual separation between benchmarks
101-121: ****The review comment incorrectly identifies a bug. The resource objects in all arrays (
scripts,styles,images,fonts,other) have aname: stringproperty—not aurlproperty. Thisnameproperty contains the resource URL (sourced fromPerformanceResourceTiming.name). The code on line 111 is using the correct property name and will successfully extract hostnames from third-party URLs.Likely an incorrect or invalid review comment.
…ing in CookieBannerCollector and improving resource timing calculations. Update bundle strategy to include bundled types and ensure robust error handling in benchmark commands. Add type safety for layout shift entries and streamline data retrieval in various collectors.
… README.md to remove 'with-' prefix from benchmark names. This change enhances consistency and simplifies command usage for running benchmarks.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 7
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
benchmarks/cookie-control/app/layout.tsx (1)
22-53: Remove redundant biome-ignore comment.The static analysis warning about dangerouslySetInnerHTML is a false positive—the injected content is a hardcoded Cookie Control configuration with no user input, making this usage safe.
However, the biome-ignore comment on line 51 is redundant and misplaced. The comment on line 22 already suppresses the lint rule for the
dangerouslySetInnerHTMLprop. The second comment before thetypeattribute serves no purpose since the lint rule doesn't apply to that attribute.Apply this diff to remove the redundant comment:
}} - // biome-ignore lint/security/noDangerouslySetInnerHtml: its okay to set inner html type="text/javascript"
♻️ Duplicate comments (4)
packages/shared/src/utils/package-manager.ts (1)
9-34: Flatten detection loop and import once.We’re still carrying the triple-nested try/catch pyramid and re-importing
node:child_processon every branch, exactly the duplication that was flagged earlier. Please flatten the control flow by importing once and iterating the candidates—this keeps the logic readable and avoids redundant dynamic imports.export async function getPackageManager(): Promise<{ command: string; args: string[]; }> { - try { - const { execSync } = await import("node:child_process"); - const output = execSync("npm -v", { encoding: "utf-8" }); - if (output) { - return { command: "npm", args: ["run"] }; - } - } catch { - try { - const { execSync } = await import("node:child_process"); - const output = execSync("yarn -v", { encoding: "utf-8" }); - if (output) { - return { command: "yarn", args: [] }; - } - } catch { - try { - const { execSync } = await import("node:child_process"); - const output = execSync("pnpm -v", { encoding: "utf-8" }); - if (output) { - return { command: "pnpm", args: [] }; - } - } catch { - // Default to npm if no package manager is found - return { command: "npm", args: ["run"] }; - } - } - } - // Fallback if all checks succeed but output is falsy (shouldn't happen in practice) - return { command: "npm", args: ["run"] }; + const { execSync } = await import("node:child_process"); + const candidates = [ + { check: "npm -v", result: { command: "npm", args: ["run"] } }, + { check: "yarn -v", result: { command: "yarn", args: [] } }, + { check: "pnpm -v", result: { command: "pnpm", args: [] } }, + ]; + + for (const candidate of candidates) { + try { + const output = execSync(candidate.check, { encoding: "utf-8" }); + if (output) { + return candidate.result; + } + } catch { + // try next candidate + } + } + + return { command: "npm", args: ["run"] }; }benchmarks/cookie-control/next-env.d.ts (1)
3-3: Duplicate concern: Manual edit of auto-generated file.This file exhibits the same pattern as
benchmarks/c15t-react/next-env.d.tsandbenchmarks/iubenda/next-env.d.ts, where route types are being manually imported into an auto-generated Next.js environment file. Please refer to the verification steps outlined in the review of those files.packages/cookiebench-cli/src/commands/scores.ts (1)
209-256: Guard against empty result sets before averaging.If a results.json exists but
resultsis empty, every average below divides by0, producingNaNscores. Bail out early so we don’t feed invalid data intocalculateScores.- const appResults = result.results; + const appResults = result.results; + if (!Array.isArray(appResults) || appResults.length === 0) { + logger.error(`No iterations recorded for ${appName}; cannot compute scores.`); + return; + }packages/benchmark/src/perfume-collector.ts (1)
248-251: Fix TTFB fallback to keep legitimate zero measurements.
||treats a0 msTTFB reading as falsy and replaces it with the navigation timing fallback, overstating ultra-fast responses. Use??so only missing metrics fall back.timeToFirstByte: - rawMetrics.TTFB?.value || navigationTiming?.timeToFirstByte || 0, + rawMetrics.TTFB?.value ?? + navigationTiming?.timeToFirstByte ?? + 0,
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (12)
benchmarks/c15t-nextjs/app/favicon.icois excluded by!**/*.icobenchmarks/c15t-react/app/favicon.icois excluded by!**/*.icobenchmarks/cookie-control/app/favicon.icois excluded by!**/*.icobenchmarks/cookie-yes/app/favicon.icois excluded by!**/*.icobenchmarks/didomi/app/favicon.icois excluded by!**/*.icobenchmarks/enzuzo/app/favicon.icois excluded by!**/*.icobenchmarks/iubenda/app/favicon.icois excluded by!**/*.icobenchmarks/ketch/app/favicon.icois excluded by!**/*.icobenchmarks/onetrust/app/favicon.icois excluded by!**/*.icobenchmarks/osano/app/favicon.icois excluded by!**/*.icobenchmarks/usercentrics/app/favicon.icois excluded by!**/*.icopnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (60)
benchmarks/c15t-nextjs/config.json(1 hunks)benchmarks/c15t-nextjs/next-env.d.ts(1 hunks)benchmarks/c15t-nextjs/package.json(2 hunks)benchmarks/c15t-react/config.json(1 hunks)benchmarks/c15t-react/next-env.d.ts(1 hunks)benchmarks/c15t-react/package.json(2 hunks)benchmarks/cookie-control/app/layout.tsx(3 hunks)benchmarks/cookie-control/config.json(1 hunks)benchmarks/cookie-control/next-env.d.ts(1 hunks)benchmarks/cookie-control/package.json(2 hunks)benchmarks/cookie-yes/config.json(2 hunks)benchmarks/cookie-yes/next-env.d.ts(1 hunks)benchmarks/cookie-yes/package.json(2 hunks)benchmarks/didomi/app/layout.tsx(3 hunks)benchmarks/didomi/config.json(1 hunks)benchmarks/didomi/next-env.d.ts(1 hunks)benchmarks/didomi/package.json(2 hunks)benchmarks/enzuzo/app/layout.tsx(2 hunks)benchmarks/enzuzo/config.json(1 hunks)benchmarks/enzuzo/next-env.d.ts(1 hunks)benchmarks/enzuzo/package.json(1 hunks)benchmarks/iubenda/app/layout.tsx(2 hunks)benchmarks/iubenda/config.json(2 hunks)benchmarks/iubenda/next-env.d.ts(1 hunks)benchmarks/iubenda/package.json(2 hunks)benchmarks/ketch/config.json(2 hunks)benchmarks/ketch/next-env.d.ts(1 hunks)benchmarks/ketch/package.json(2 hunks)benchmarks/onetrust/app/layout.tsx(2 hunks)benchmarks/onetrust/config.json(2 hunks)benchmarks/onetrust/next-env.d.ts(1 hunks)benchmarks/onetrust/package.json(2 hunks)benchmarks/osano/config.json(2 hunks)benchmarks/osano/next-env.d.ts(1 hunks)benchmarks/osano/next.config.ts(1 hunks)benchmarks/osano/package.json(1 hunks)benchmarks/osano/tsconfig.json(1 hunks)benchmarks/usercentrics/config.json(2 hunks)benchmarks/usercentrics/next-env.d.ts(1 hunks)benchmarks/usercentrics/next.config.ts(1 hunks)benchmarks/usercentrics/package.json(2 hunks)benchmarks/usercentrics/tsconfig.json(1 hunks)benchmarks/with-enzuzo/next-env.d.ts(0 hunks)benchmarks/with-iubenda/next-env.d.ts(0 hunks)benchmarks/with-ketch/next-env.d.ts(0 hunks)benchmarks/with-onetrust/next-env.d.ts(0 hunks)benchmarks/with-osano/next-env.d.ts(0 hunks)benchmarks/with-usercentrics/next-env.d.ts(0 hunks)packages/benchmark/src/bundle-strategy.ts(1 hunks)packages/benchmark/src/cookie-banner-collector.ts(1 hunks)packages/benchmark/src/perfume-collector.ts(1 hunks)packages/benchmark/src/resource-timing-collector.ts(1 hunks)packages/benchmark/src/types.ts(1 hunks)packages/cookiebench-cli/README.md(1 hunks)packages/cookiebench-cli/src/commands/benchmark.ts(1 hunks)packages/cookiebench-cli/src/commands/scores.ts(1 hunks)packages/runner/src/performance-aggregator.ts(1 hunks)packages/runner/src/types.ts(1 hunks)packages/shared/src/utils/package-manager.ts(1 hunks)packages/shared/src/utils/time.ts(1 hunks)
💤 Files with no reviewable changes (6)
- benchmarks/with-usercentrics/next-env.d.ts
- benchmarks/with-onetrust/next-env.d.ts
- benchmarks/with-osano/next-env.d.ts
- benchmarks/with-enzuzo/next-env.d.ts
- benchmarks/with-ketch/next-env.d.ts
- benchmarks/with-iubenda/next-env.d.ts
🧰 Additional context used
🧬 Code graph analysis (11)
packages/shared/src/utils/package-manager.ts (3)
packages/cookiebench-cli/src/utils/index.ts (1)
getPackageManager(10-10)packages/runner/src/index.ts (1)
getPackageManager(31-31)packages/shared/src/index.ts (1)
getPackageManager(21-21)
packages/benchmark/src/resource-timing-collector.ts (2)
packages/benchmark/src/types.ts (1)
ResourceTimingData(129-214)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(17-74)
packages/benchmark/src/perfume-collector.ts (1)
packages/benchmark/src/types.ts (2)
WindowWithPerfumeMetrics(264-280)PerfumeMetrics(231-262)
packages/cookiebench-cli/src/commands/scores.ts (7)
packages/cookiebench-cli/src/commands/results.ts (2)
BenchmarkOutput(179-213)RawBenchmarkDetail(37-177)packages/cookiebench-cli/src/types/index.ts (1)
BenchmarkScores(10-38)packages/cookiebench-cli/src/utils/logger.ts (2)
logger(162-162)CliLogger(9-9)packages/benchmark/src/types.ts (1)
Config(13-56)packages/shared/src/constants.ts (2)
HALF_SECOND(3-3)PERCENTAGE_DIVISOR(11-11)packages/cookiebench-cli/src/utils/scoring.ts (2)
printScores(1383-1422)calculateScores(1010-1379)packages/cookiebench-cli/src/utils/constants.ts (1)
DEFAULT_DOM_SIZE(3-3)
packages/shared/src/utils/time.ts (4)
packages/cookiebench-cli/src/utils/index.ts (2)
formatTime(9-9)ONE_SECOND(13-13)packages/runner/src/index.ts (1)
formatTime(31-31)packages/shared/src/index.ts (2)
formatTime(22-22)ONE_SECOND(8-8)packages/shared/src/constants.ts (1)
ONE_SECOND(2-2)
packages/benchmark/src/cookie-banner-collector.ts (3)
packages/benchmark/src/types.ts (5)
Config(13-56)CookieBannerMetrics(80-93)WindowWithCookieMetrics(65-78)LayoutShiftEntry(59-62)CookieBannerData(95-104)packages/benchmark/src/bundle-strategy.ts (1)
determineBundleStrategy(4-25)packages/benchmark/src/constants.ts (1)
BENCHMARK_CONSTANTS(17-74)
packages/runner/src/performance-aggregator.ts (4)
packages/benchmark/src/types.ts (7)
CookieBannerData(95-104)CookieBannerMetrics(80-93)NetworkRequest(107-114)NetworkMetrics(116-119)ResourceTimingData(129-214)Config(13-56)PerfumeMetrics(231-262)packages/runner/src/types.ts (9)
CookieBannerData(8-8)CookieBannerMetrics(9-9)NetworkRequest(12-12)NetworkMetrics(11-11)ResourceTimingData(14-14)Config(6-6)PerfumeMetrics(13-13)BenchmarkDetails(24-159)BenchmarkResult(161-274)packages/shared/src/constants.ts (2)
TTI_BUFFER_MS(14-14)PERCENTAGE_MULTIPLIER(10-10)packages/runner/src/statistics.ts (4)
isStable(132-138)calculateCoefficientOfVariation(112-127)calculateTrimmedMean(84-107)calculateStatistics(11-52)
benchmarks/enzuzo/app/layout.tsx (9)
benchmarks/cookie-control/app/layout.tsx (1)
metadata(4-6)benchmarks/c15t-react/app/layout.tsx (1)
metadata(16-18)benchmarks/iubenda/app/layout.tsx (1)
metadata(4-6)benchmarks/cookie-yes/app/layout.tsx (1)
metadata(4-6)benchmarks/c15t-nextjs/app/layout.tsx (1)
metadata(16-18)benchmarks/onetrust/app/layout.tsx (1)
metadata(4-6)benchmarks/osano/app/layout.tsx (1)
metadata(4-6)benchmarks/ketch/app/layout.tsx (1)
metadata(4-6)benchmarks/usercentrics/app/layout.tsx (1)
metadata(4-6)
packages/benchmark/src/bundle-strategy.ts (2)
packages/benchmark/src/types.ts (2)
Config(13-56)BundleStrategy(122-126)packages/benchmark/src/constants.ts (1)
BUNDLE_TYPES(76-81)
packages/runner/src/types.ts (2)
packages/runner/src/index.ts (3)
ServerInfo(21-21)BenchmarkDetails(10-10)BenchmarkResult(11-11)packages/cookiebench-cli/src/types/index.ts (3)
ServerInfo(6-6)BenchmarkDetails(3-3)BenchmarkResult(4-4)
packages/cookiebench-cli/src/commands/benchmark.ts (6)
packages/runner/src/types.ts (2)
BenchmarkResult(161-274)ServerInfo(18-21)packages/runner/src/benchmark-runner.ts (2)
runSingleBenchmark(96-199)BenchmarkRunner(30-474)packages/shared/src/utils/config.ts (1)
readConfig(14-26)packages/runner/src/server.ts (2)
buildAndServeNextApp(6-71)cleanupServer(73-77)packages/cookiebench-cli/src/utils/scoring.ts (2)
calculateScores(1010-1379)printScores(1383-1422)packages/cookiebench-cli/src/commands/results.ts (1)
resultsCommand(890-1120)
🪛 ast-grep (0.39.6)
benchmarks/iubenda/app/layout.tsx
[warning] 18-18: Usage of dangerouslySetInnerHTML detected. This bypasses React's built-in XSS protection. Always sanitize HTML content using libraries like DOMPurify before injecting it into the DOM to prevent XSS attacks.
Context: dangerouslySetInnerHTML
Note: [CWE-79] Improper Neutralization of Input During Web Page Generation [REFERENCES]
- https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml
- https://cwe.mitre.org/data/definitions/79.html
(react-unsafe-html-injection)
benchmarks/onetrust/app/layout.tsx
[warning] 23-23: Usage of dangerouslySetInnerHTML detected. This bypasses React's built-in XSS protection. Always sanitize HTML content using libraries like DOMPurify before injecting it into the DOM to prevent XSS attacks.
Context: dangerouslySetInnerHTML
Note: [CWE-79] Improper Neutralization of Input During Web Page Generation [REFERENCES]
- https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml
- https://cwe.mitre.org/data/definitions/79.html
(react-unsafe-html-injection)
benchmarks/cookie-control/app/layout.tsx
[warning] 22-22: Usage of dangerouslySetInnerHTML detected. This bypasses React's built-in XSS protection. Always sanitize HTML content using libraries like DOMPurify before injecting it into the DOM to prevent XSS attacks.
Context: dangerouslySetInnerHTML
Note: [CWE-79] Improper Neutralization of Input During Web Page Generation [REFERENCES]
- https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml
- https://cwe.mitre.org/data/definitions/79.html
(react-unsafe-html-injection)
🪛 LanguageTool
packages/cookiebench-cli/README.md
[uncategorized] ~373-~373: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ice overhead 4. Transparency (15%): Open source status, documentation, licensing 5. **U...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🪛 markdownlint-cli2 (0.18.1)
packages/cookiebench-cli/README.md
99-99: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
99-99: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
170-170: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
170-170: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
336-336: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
🔇 Additional comments (70)
benchmarks/cookie-yes/package.json (3)
2-2: Naming consistency with refactoring objectives.The package name change from "with-cookie-yes" to "cookie-yes" aligns with the consolidation theme and shorter naming convention in the PR. This change appears consistent with other benchmark project renames mentioned in the PR context.
22-22: Verify dependency replacement preserves functionality.The change from
@cookiebench/clitocookiebenchaligns with the new centralized CLI tooling described in the PR. However, confirm that this replacement provides equivalent or enhanced functionality and that any build scripts or workflows dependent on the old CLI remain compatible.
25-27: Node engine constraint aligns with workspace standards.The addition of
engines: { "node": ">=20.9.0" }is consistent with other benchmark packages in the PR. Verify that this minimum version is compatible with all declared dependencies, particularlynext@16.0.1,react@^19.2.0, andtypescript@^5.9.3.benchmarks/osano/next.config.ts (1)
1-1: LGTM — consistent quote style update.This formatting adjustment aligns the import statement with double-quote conventions, likely as part of the broader linting/formatting standardization introduced in this PR (new
.cursor/rules/ultracite.mdc). No functional changes.benchmarks/cookie-yes/config.json (4)
3-3: Project name updated to match naming convention.The name has been changed from
with-cookie-yestocookie-yes, aligning with the broader refactor. This change is consistent with the PR objectives.
8-8: Remote URL updated to reflect the new project name.The URL has been updated from
benchmarks-with-cookie-yes.vercel.apptobenchmarks-cookie-yes.vercel.app, maintaining consistency with the name change.Please verify that the new Vercel deployment URL is accessible and correctly configured to serve the benchmark.
24-24: Typo correction: "iffe" → "iife".The bundle type has been corrected from
"iffe"(a typo) to"iife"(Immediately Invoked Function Expression), which is the correct term.
4-4: Verify the ID field consistency.The
idfield remains"cookie-yes-banner"while thenamefield has changed to"cookie-yes". Confirm that this ID is not used elsewhere in the codebase in ways that would create inconsistencies, and that the differing naming conventions are intentional.benchmarks/didomi/package.json (3)
2-2: Package name alignment consistent with project renaming convention.The name field has been updated to
"didomi", removing the"with-"prefix and aligning with the standardized naming convention across the benchmarks suite as part of this refactoring.
4-10: Scripts section appropriately updated for new tooling.The removal of the public
"benchmark"script aligns with the refactoring to use the newcookiebench-clitooling instead of individual package scripts. The retained scripts (build,dev,fmt,lint,start) are appropriate for a Next.js benchmark project.
25-27: ✅ Engines constraint properly specified (resolves previous critical issue).The
"engines"field is correctly specified using the plural form (not the singular"engine"), which ensures that package managers will properly enforce the Node.js runtime constraint. This resolves the critical issue flagged in previous reviews and aligns with the standardized engine requirements across the benchmarks suite.benchmarks/didomi/config.json (1)
3-3: Name standardization is consistent and verified across all benchmarks.Verification confirms:
- No remaining references to the old "with-didomi" name in the codebase
- All other benchmark configs consistently use the same naming pattern without the "with-" prefix (e.g., "iubenda", "onetrust", "usercentrics", "osano", "ketch", "cookie-control", "enzuzo", "c15t-react", "baseline", "cookie-yes")
- The change aligns with the standardized naming convention for the benchmark suite
benchmarks/enzuzo/app/layout.tsx (1)
1-2: LGTM! Formatting improvements for consistency.The quote style changes and indentation adjustments improve consistency with other benchmark layout files across the codebase.
Also applies to: 5-5, 18-18
benchmarks/cookie-yes/next-env.d.ts (1)
3-3: LGTM! Typed routes import added correctly.This import enables Next.js's typed routes feature, which provides type-safe routing paths. The addition aligns with Next.js 15.5's stable typed routes support and is consistent with the pattern applied across other benchmarks in this PR.
Based on library documentation
benchmarks/enzuzo/package.json (1)
25-27: ✅ Critical issue resolved:enginesfield now correctly named.The previous critical issue flagging
"engine"(singular) has been properly corrected to"engines"(plural) on the package.json spec. Package managers will now correctly read the Node.js version constraint.benchmarks/enzuzo/config.json (2)
7-7: Verify the cookie banner selector against Enzuzo's current implementation.The selector
".enzuzo-cookiebanner-container"(line 7) controls which DOM element is measured during benchmarking. If Enzuzo's banner HTML structure changes or if this selector is incorrect, measurements will fail or return invalid results.Please confirm that this selector correctly targets Enzuzo's cookie banner container in their current implementation. You may want to test this locally or verify against Enzuzo's documentation/source.
2-2: Verify schema reference resolves correctly.The schema reference on line 2 points to
./node_modules/@cookiebench/benchmark-schema/schema.json. Ensure this package is properly installed as a dependency and that the schema file exists at this path during both development and CI/CD environments.benchmarks/cookie-control/package.json (3)
2-2: Package rename improves clarity.Renaming from "with-enzuzo" to "cookie-control" better reflects the benchmark's purpose and aligns with the new naming convention applied across benchmarks.
Also applies to: 2-2
22-22: Verifycookiebenchworkspace dependency replaces CLI functionality.The change from
@cookiebench/clitocookiebenchis part of the CLI refactoring. Ensure the new dependency provides all tooling previously available via the benchmark script that was removed.Also applies to: 22-22
25-27: Node engine constraint added consistently.The engines constraint
node: >=20.9.0aligns with the Next.js 16.0.1 and React 19.2.0 requirements and should be compatible across the benchmark suite. Verify this minimum version is enforced in CI/deployment.benchmarks/cookie-control/config.json (1)
1-40: New benchmark configuration file is well-structured.The configuration correctly captures Cookie Control's metadata, cookie banner detection specifics (proprietary selector
.ccc-module--slideout, Civic Computing service host), and appropriate tech stack details (IIFE bundleType for proprietary delivery, no TypeScript, no frameworks). The schema reference ensures consistency with the benchmark-schema contract.Verify the schema at
./node_modules/@cookiebench/benchmark-schema/schema.jsonexists and validates this configuration during build/test.benchmarks/c15t-nextjs/package.json (3)
2-2: Package rename follows consistent convention.Renaming from "with-c15t-nextjs" to "c15t-nextjs" matches the pattern applied to other benchmarks and improves naming clarity.
Also applies to: 2-2
24-24: Consistent CLI dependency refactoring.The change to
cookiebenchworkspace dependency mirrors the pattern in other benchmarks and aligns with the PR's CLI consolidation objective. Ensure the new package provides equivalent functionality.Also applies to: 24-24
27-29: Engine constraint applied consistently.Node >=20.9.0 matches the requirement applied across the benchmark suite and is compatible with the Next.js and React versions pinned in this package.
packages/shared/src/utils/time.ts (1)
8-20: LGTM! Previous feedback fully addressed.The implementation correctly handles all edge cases suggested in the previous review (non-finite values, negative numbers) and provides appropriate formatting for both milliseconds and seconds. The validation logic is clean and the rounding behavior is suitable for display purposes.
benchmarks/iubenda/app/layout.tsx (2)
18-18: Lint suppression is appropriate.The biome-ignore comment correctly justifies the use of
dangerouslySetInnerHTMLsince the content is entirely hardcoded configuration with no user input. The static analysis warning is a false positive in this case.
82-85: Question: Is this attribute reordering intentional?The
srcattribute was moved to appear aftercharSet. While this has no functional impact, it adds noise to the diff. Was this change required by the new Ultracite Rules configuration mentioned in the PR, or was it unintentional?benchmarks/cookie-control/app/layout.tsx (1)
1-5: LGTM: Formatting consistency improvements.The quote style changes align with the broader refactoring effort for consistent conventions across the codebase.
benchmarks/c15t-nextjs/config.json (1)
3-3: The name change is correct and safe.Verification confirms:
- No references to the old name "with-c15t-nextjs" exist in the codebase
- Naming convention "c15t-nextjs" is consistent with all other benchmarks (c15t-react, baseline, usercentrics, etc.)
- The cookiebench CLI and turbo.json use generic task configurations that don't depend on specific benchmark names
- This aligns with the PR's normalization goals
benchmarks/onetrust/package.json (1)
25-27: Node version constraint is consistent across all benchmark packages—no action required.The verification confirms that all 12 benchmark packages in the workspace already enforce
engines.node >= 20.9.0uniformly, including onetrust. This consistency is maintained across the entire workspace, and the constraint aligns with the dependencies used (React, TypeScript, and related types).benchmarks/iubenda/package.json (2)
2-2: Name consistency and engine constraints look good.The rename from "with-cookie-yes" to "iubenda" aligns with the PR's consistent naming objective, and the Node engine constraint (>=20.9.0) is appropriately aligned with the broader benchmark infrastructure updates.
Also applies to: 25-27
4-9: No changes needed; benchmark refactoring is correct.The PR properly refactors benchmark automation. The
benchmarks/iubendapackage.json correctly:
- Removes its local
"benchmark"script (now delegated to root)- Swaps
@cookiebench/cliforcookiebenchin devDependencies- The
cookiebenchpackage (packages/cookiebench-cli) confirms it exports CLI functionality via its bin entry:"cookiebench": "dist/index.mjs"- Root package.json retains the
"benchmark"script invoking"pnpm exec cookiebench benchmark", ensuring CI/automation workflows continue to functionbenchmarks/c15t-react/config.json (1)
3-3: Consistent naming normalization.The benchmark name is being normalized from "with-c15t-react" to "c15t-react", aligning with the PR's broader standardization effort.
benchmarks/c15t-react/package.json (4)
2-2: Package name normalized consistently.The package name matches the config.json update, maintaining consistency across the benchmark configuration.
27-29: Node.js engine constraint correctly specified.The
enginesfield now uses the correct plural form (fixing the previousenginetypo). The Node.js version constraint (>=20.9.0) is appropriate for this project.
24-24: Dependency migration is valid.The
cookiebenchworkspace reference is correct. The package exists atpackages/cookiebench-cli/with the namecookiebenchand provides CLI functionality via the bin entry"cookiebench": "dist/index.mjs". The pnpm workspace configuration properly includes this package.
4-9: Review comment is based on incorrect assumptions about the change.The
benchmarks/c15t-react/package.jsonfile is a new file being added to the repository, not a modified file. The git diff showsnew file mode 100644, and this is the first commit touching this file. There was no previous version to remove any script from.The architecture is correct: individual benchmark packages like
c15t-reactdo not define their ownbenchmarkscripts. Instead, benchmark execution is managed centrally via the rootpackage.jsonusing thecookiebenchCLI (which is already available as a workspace dependency in this file'sdevDependencies). This is consistent across all other benchmarks in the repository.No action is required.
Likely an incorrect or invalid review comment.
benchmarks/onetrust/app/layout.tsx (1)
16-21: No issues found — async attribute is correct per OneTrust best practices.The
asyncattribute onotSDKStub.jsis the recommended pattern according to OneTrust documentation. The synchronousOptanonWrappercallback defined immediately after ensures proper initialization timing. The implementation is correct as-is.benchmarks/didomi/app/layout.tsx (3)
32-34: LGTM!The multi-line early return improves readability without changing functionality.
58-59: LGTM!Prop reordering has no functional impact in JSX.
61-63: LGTM!The event handlers are appropriately reintroduced with consistent console logging for demonstration purposes in this benchmark application.
benchmarks/iubenda/config.json (2)
3-3: LGTM: Package naming standardization.The removal of the "with-" prefix aligns with the broader refactoring effort to standardize benchmark naming conventions across the repository.
25-25: Excellent: Typo correction for bundleType.The change from "iffe" to "iife" (Immediately Invoked Function Expression) corrects an important typo. This ensures the configuration accurately reflects the bundle type being used.
benchmarks/osano/config.json (1)
3-3: LGTM: Naming standardization and typo correction.Both changes align with the broader refactoring:
- Removing the "with-" prefix standardizes benchmark naming
- Correcting "iffe" → "iife" fixes the bundle type designation (IIFE = Immediately Invoked Function Expression)
These changes mirror the pattern applied consistently across other benchmark configurations in this PR.
Also applies to: 16-16
benchmarks/usercentrics/tsconfig.json (1)
1-11: LGTM: Standard TypeScript configuration.The configuration appropriately extends the shared Next.js TypeScript config and sets up standard path aliases and compilation options. This aligns with best practices for Next.js projects.
benchmarks/usercentrics/next.config.ts (1)
1-1: LGTM: Quote style standardization.The change from single to double quotes is a cosmetic formatting adjustment with no semantic impact, likely part of broader code style standardization across the repository.
benchmarks/usercentrics/package.json (3)
2-2: LGTM: Package naming standardization.The package rename from "with-usercentrics" to "usercentrics" is consistent with the broader naming standardization effort across all benchmark packages in this PR.
22-22: LGTM: CLI tooling migration.The replacement of
@cookiebench/cliwithcookiebenchaligns with the PR's objective to replace the old CLI with a newcookiebench-clipackage as part of the infrastructure reorganization.
25-27: Address undocumented Node.js engine constraint and clarify versioning strategy.The benchmark packages consistently use
>=20.9.0, but this requires verification:
- Documentation gap: Node.js version requirement is not documented in README or contribution guidelines.
- Misalignment with main repo: Main package.json specifies
>=18, while benchmarks require>=20.9.0—clarify if this intentional separation is necessary or should be aligned.- Justification unclear: The constraint is more restrictive than the main repo without documented rationale (Next 16.0.1 and React 19.2.0 don't inherently require 20.9.0).
Ensure the engine constraint is either: (a) documented in contribution guidelines with clear rationale, or (b) aligned with main repository requirements if the higher version is not strictly necessary.
benchmarks/iubenda/next-env.d.ts (1)
1-6: Remove incorrect import from auto-generatednext-env.d.tsfile.When
typedRoutesis enabled in Next.js configuration, it generates a.next/types/link.d.tsfile—notroutes.js. Since none of the benchmarks havetypedRoutesconfigured, the importimport "./.next/types/routes.js"references a non-existent file and should be removed frombenchmarks/iubenda/next-env.d.ts(and any other affected files). This import violates the file's own directive stating it "should not be edited."If typed routes are intended for these benchmarks, enable
typedRoutes: truein each benchmark'snext.config.tsinstead of manually editing the generated file.Likely an incorrect or invalid review comment.
benchmarks/c15t-nextjs/next-env.d.ts (1)
3-3: LGTM! Typed routes import added correctly.This import enables Next.js 15's typed routes feature, which is stable as of version 15.5. The pattern is consistent across all benchmarks in this PR.
benchmarks/onetrust/config.json (2)
3-3: LGTM! Benchmark name normalized.The name change from "with-onetrust" to "onetrust" aligns with the broader naming convention standardization across benchmarks in this PR.
22-22: LGTM! Typo corrected.The correction from "iffe" to "iife" fixes the typo for the standard acronym (Immediately Invoked Function Expression).
benchmarks/osano/next-env.d.ts (1)
1-6: LGTM! Standard Next.js type declarations.This file follows the standard Next.js pattern for TypeScript environment setup, including typed routes support. The content is consistent with other benchmarks in the PR.
benchmarks/ketch/config.json (2)
3-3: LGTM! Benchmark name normalized.The name change from "with-ketch" to "ketch" aligns with the broader naming convention standardization across benchmarks in this PR.
19-19: LGTM! Typo corrected.The correction from "iffe" to "iife" fixes the typo for the standard acronym (Immediately Invoked Function Expression).
benchmarks/didomi/next-env.d.ts (1)
1-6: LGTM! Standard Next.js type declarations.This file follows the standard Next.js pattern for TypeScript environment setup, including typed routes support. The content is consistent with other benchmarks in the PR.
benchmarks/onetrust/next-env.d.ts (1)
1-6: LGTM! Standard Next.js type declarations.This file follows the standard Next.js pattern for TypeScript environment setup, including typed routes support. The content is consistent with other benchmarks in the PR.
benchmarks/enzuzo/next-env.d.ts (1)
1-6: LGTM! Standard Next.js type declarations.This file follows the standard Next.js pattern for TypeScript environment setup, including typed routes support. The content is consistent with other benchmarks in the PR.
benchmarks/ketch/package.json (2)
2-2: LGTM! Package name normalized.The name change from "with-ketch" to "ketch" is consistent with the corresponding config.json change and aligns with the broader naming convention standardization across benchmarks in this PR.
24-26: LGTM! Node.js version constraint added correctly.The
enginesfield is correctly spelled (plural) and specifies a reasonable Node.js version constraint. The previous typo issue flagged in past reviews has been properly resolved.benchmarks/usercentrics/config.json (2)
3-3: LGTM: Name simplification improves consistency.The name change from "with-usercentrics" to "usercentrics" aligns with the PR's goal of standardizing benchmark naming conventions.
20-20: LGTM: Typo correction.The change from "iffe" to "iife" correctly represents the bundle type (Immediately Invoked Function Expression).
benchmarks/osano/tsconfig.json (1)
1-11: LGTM: Standard Next.js TypeScript configuration.The configuration appropriately extends the shared cookiebench TypeScript config and uses standard Next.js patterns for path resolution and file inclusion.
benchmarks/usercentrics/next-env.d.ts (1)
1-6: LGTM: Standard Next.js environment type declarations.This file correctly establishes ambient TypeScript types for Next.js. The pattern is consistent with other benchmarks in this PR, which is good for maintainability.
benchmarks/osano/package.json (1)
1-28: LGTM: Package configuration is correct.The package manifest is well-structured with appropriate dependencies and scripts. Note that the previous critical issue with "engine" vs "engines" has been addressed in commit a823f20, and the current configuration correctly uses "engines".
benchmarks/ketch/next-env.d.ts (1)
1-6: LGTM: Standard Next.js environment type declarations.This file correctly establishes ambient TypeScript types for Next.js, maintaining consistency with other benchmark projects in this PR.
packages/benchmark/src/types.ts (1)
1-280: Comprehensive and well-structured type definitions.The type definitions are thorough and properly organized. The Window augmentations (
WindowWithCookieMetrics,WindowWithPerfumeMetrics) correctly extend the Window interface, and the LayoutShiftEntry interface properly extends PerformanceEntry. All types provide clear contracts for the benchmark infrastructure and align well with their usage in the collector implementations.packages/benchmark/src/cookie-banner-collector.ts (2)
121-145: Excellent improvements to type safety and resource management.The addition of the
isLayoutShiftEntrytype guard (lines 121-128) properly validates entry properties before type narrowing, and storing the observer reference in__cookieBannerMetrics.clsObserver(lines 143-145) enables proper cleanup later. These changes address previous feedback about type assertions and observer lifecycle management.
298-306: Proper observer cleanup prevents memory leaks.The observer disconnection logic (lines 303-306) properly cleans up the PerformanceObserver after reading final metrics, addressing the previous concern about potential memory buildup in long-running pages. Reading the layout shift value before disconnecting ensures accurate final measurements.
| @@ -1,5 +1,6 @@ | |||
| /// <reference types="next" /> | |||
| /// <reference types="next/image-types/global" /> | |||
| import "./.next/types/routes.js"; | |||
There was a problem hiding this comment.
🧩 Analysis chain
Verify: Manual edit of auto-generated file.
This file includes a comment stating it "should not be edited" (Line 5-6), yet the route types import is being manually added. In Next.js 15, with the typedRoutes config enabled, this import should be auto-generated. If you're manually adding it without enabling typed routes in next.config.ts, this could lead to:
- Type mismatches if the routes file doesn't exist or is out of sync
- The import being overwritten when Next.js regenerates this file
Please verify:
🏁 Script executed:
#!/bin/bash
# Check if typedRoutes is enabled in the Next.js config
rg -n "typedRoutes|experimental.*typedRoutes" benchmarks/c15t-react/next.config.ts benchmarks/c15t-react/next.config.js benchmarks/c15t-react/next.config.mjsLength of output: 154
🌐 Web query:
Does Next.js 15 require typedRoutes to be enabled in next.config for the .next/types/routes.js import to work?
💡 Result:
Yes. Next.js generates the .next/types route helpers only when the typed routes feature is enabled (and you’re using TypeScript). In Next.js 15 this is controlled by the typedRoutes config (previously experimental.typedRoutes); enable it (typedRoutes: true) so the .next/types/*.ts (or .js) route types are emitted. [1][2]
Sources:
- Next.js docs — typedRoutes config. [1]
- Next.js 15 release notes / TypeScript improvements (typed routes). [2]
---
🏁 Script executed:
```shell
#!/bin/bash
# Find the actual next.config file
fd "next\.config\." benchmarks/c15t-react/
Length of output: 36
🏁 Script executed:
#!/bin/bash
# Search for typedRoutes in the benchmarks directory
rg -n "typedRoutes" benchmarks/c15t-react/ -C 3Length of output: 0
🏁 Script executed:
#!/bin/bash
# View the full next-env.d.ts file to see the comment and full context
cat benchmarks/c15t-react/next-env.d.tsLength of output: 244
🏁 Script executed:
#!/bin/bash
# View the next.config.ts to confirm typedRoutes configuration
cat benchmarks/c15t-react/next.config.tsLength of output: 131
Enable typedRoutes in next.config.ts or remove the manual import.
The next-env.d.ts file explicitly warns against manual editing, yet line 3 contains a manually added import to ./.next/types/routes.js. Next.js generates the .next/types route helpers only when the typed routes feature is enabled, and typedRoutes is not currently configured in next.config.ts (benchmarks/c15t-react/next.config.ts is empty). Without this configuration, the import will either fail at runtime or be overwritten when Next.js regenerates the file. Add typedRoutes: true to the nextConfig object in next.config.ts.
🤖 Prompt for AI Agents
In benchmarks/c15t-react/next-env.d.ts around line 3 there is a manual import of
"./.next/types/routes.js" which is unsafe because typed route helpers are only
generated when typedRoutes is enabled; update
benchmarks/c15t-react/next.config.ts to add typedRoutes: true to the exported
nextConfig object (or remove the manual import from next-env.d.ts) so Next.js
will generate .next/types/routes.js consistently; ensure the change is in the
nextConfig export and committed so the import will not fail or be overwritten.
| @@ -1,3 +1,4 @@ | |||
| /** biome-ignore-all lint/suspicious/noConsole: its okay to show it working */ | |||
There was a problem hiding this comment.
Incorrect comment syntax for Biome suppression.
Biome file-level suppression comments must start with biome-ignore-all, but Biome uses // comment style for suppression comments, not JSDoc-style block comments. The current JSDoc-style comment (/** ... */) is non-standard and may not work as intended.
Apply this diff to use the correct comment syntax:
-/** biome-ignore-all lint/suspicious/noConsole: its okay to show it working */
+// biome-ignore-all lint/suspicious/noConsole: its okay to show it working🤖 Prompt for AI Agents
In benchmarks/didomi/app/layout.tsx around line 1, the Biome suppression is
currently written as a JSDoc block comment (/** biome-ignore-all ... */) which
Biome does not recognize; replace it with a line comment using the correct
syntax: start the file with // biome-ignore-all lint/suspicious/noConsole: its
okay to show it working so the file-level suppression is applied properly.
| src="https://cdn.cookielaw.org/scripttemplates/otSDKStub.js" | ||
| /> | ||
| <script | ||
| // biome-ignore lint/security/noDangerouslySetInnerHtml: its okay to set inner html |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Improve the biome-ignore comment justification.
The comment reason "its okay to set inner html" is vague. Consider a more descriptive justification that explains why this usage is safe, such as: "Required OneTrust callback stub - hardcoded content with no user input".
Apply this diff to improve the comment:
- // biome-ignore lint/security/noDangerouslySetInnerHtml: its okay to set inner html
+ // biome-ignore lint/security/noDangerouslySetInnerHtml: Required OneTrust callback stub - hardcoded content with no user input📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| // biome-ignore lint/security/noDangerouslySetInnerHtml: its okay to set inner html | |
| // biome-ignore lint/security/noDangerouslySetInnerHtml: Required OneTrust callback stub - hardcoded content with no user input |
🧰 Tools
🪛 ast-grep (0.39.6)
[warning] 23-23: Usage of dangerouslySetInnerHTML detected. This bypasses React's built-in XSS protection. Always sanitize HTML content using libraries like DOMPurify before injecting it into the DOM to prevent XSS attacks.
Context: dangerouslySetInnerHTML
Note: [CWE-79] Improper Neutralization of Input During Web Page Generation [REFERENCES]
- https://reactjs.org/docs/dom-elements.html#dangerouslysetinnerhtml
- https://cwe.mitre.org/data/definitions/79.html
(react-unsafe-html-injection)
🤖 Prompt for AI Agents
In benchmarks/onetrust/app/layout.tsx around line 23, the biome-ignore comment
justification is vague; replace the current "its okay to set inner html" with a
more descriptive reason such as "Required OneTrust callback stub - hardcoded
content with no user input" so the linter exception documents why
dangerouslySetInnerHTML is safe here and clarifies there is no user-supplied
content involved.
| ``` | ||
| ? Select benchmarks to run: | ||
| ◼ baseline | ||
| ◼ c15t-nextjs | ||
| ◼ cookieyes | ||
|
|
||
| ● info Config iterations: baseline: 5, c15t-nextjs: 3, cookieyes: 5 | ||
|
|
||
| ? Number of iterations (press Enter to use config values): | ||
| › Default: 5 | ||
|
|
||
| [Just press Enter to use config values, or type a number to override all] | ||
|
|
||
| ? Show results panel after completion? › Yes | ||
| ``` |
There was a problem hiding this comment.
Fix markdown fences to satisfy lint.
markdownlint still flags these fences because they lack surrounding blank lines and language identifiers. Add the empty lines before/after each block and declare an appropriate language so the docs build doesn’t fail.
-**Example:**
-```
+**Example:**
+
+```text
? Select benchmarks to run:
◼ baseline
◼ c15t-nextjs
◼ cookieyes
-```
+```
-**Interactive Multi-Select Mode:**
-```
+**Interactive Multi-Select Mode:**
+
+```text
? Select benchmarks to view (use space to toggle, all selected by default):
◼ baseline (benchmarks/baseline)
◼ c15t-nextjs (benchmarks/c15t-nextjs)
◼ cookieyes (benchmarks/cookieyes)
-```
+```
-**For Consent.io team members:**
-```
+**For Consent.io team members:**
+
+```bash
# Enable admin mode
export CONSENT_ADMIN=true
@@
cookiebench save
cookiebench db push
-```
+```Also applies to: 170-178, 336-343
🧰 Tools
🪛 markdownlint-cli2 (0.18.1)
99-99: Fenced code blocks should be surrounded by blank lines
(MD031, blanks-around-fences)
99-99: Fenced code blocks should have a language specified
(MD040, fenced-code-language)
🤖 Prompt for AI Agents
In packages/cookiebench-cli/README.md around lines 99-113 (also apply same fixes
at 170-178 and 336-343): the fenced code blocks lack surrounding blank lines and
language identifiers causing markdownlint failures; add a blank line before and
after each triple-backtick fence and add an appropriate language tag (e.g., text
for interactive prompts, bash for shell snippets) to each opening fence so every
block is surrounded by empty lines and begins with ```text or ```bash as
applicable.
| 4. **Transparency** (15%): Open source status, documentation, licensing | ||
| 5. **User Experience** (10%): Banner timing, coverage, layout shifts |
There was a problem hiding this comment.
Hyphenate the compound adjective.
When “open source” modifies “status,” markdownlint/LanguageTool expect “open-source status.” Quick wording tweak keeps automated checks quiet.
🧰 Tools
🪛 LanguageTool
[uncategorized] ~373-~373: If this is a compound adjective that modifies the following noun, use a hyphen.
Context: ...ice overhead 4. Transparency (15%): Open source status, documentation, licensing 5. **U...
(EN_COMPOUND_ADJECTIVE_INTERNAL)
🤖 Prompt for AI Agents
In packages/cookiebench-cli/README.md around lines 373 to 374, the phrase "Open
source status" should be hyphenated as "Open-source status"; update the markdown
line so the compound adjective modifying "status" uses a hyphen ("Open-source
status") to satisfy markdownlint/LanguageTool and keep automated checks quiet.
| visibilityTime: | ||
| cookieBannerData?.bannerVisibilityTime || | ||
| cookieBannerData?.bannerInteractiveTime || | ||
| 0, | ||
| viewportCoverage: cookieBannerData?.viewportCoverage || 0, |
There was a problem hiding this comment.
Preserve zero visibility timings with nullish coalescing.
The intent is to fall back to bannerInteractiveTime only when bannerVisibilityTime is absent, but || also fires when the measured visibility is 0. A banner that becomes visible at 0 ms will therefore get overwritten by the (typically higher) interactive time, inflating the reported UX timing. Please switch these fallbacks to ?? so only null/undefined trigger the fallback.
visibilityTime:
- cookieBannerData?.bannerVisibilityTime ||
- cookieBannerData?.bannerInteractiveTime ||
+ cookieBannerData?.bannerVisibilityTime ??
+ cookieBannerData?.bannerInteractiveTime ??
0,
...
visibilityTime:
- cookieBannerData?.bannerVisibilityTime ||
- cookieBannerData?.bannerInteractiveTime ||
+ cookieBannerData?.bannerVisibilityTime ??
+ cookieBannerData?.bannerInteractiveTime ??
0,Also applies to: 206-210
🤖 Prompt for AI Agents
In packages/runner/src/performance-aggregator.ts around lines 90-94 (and
similarly update lines 206-210), the visibilityTime and viewportCoverage
fallbacks use || which treats 0 as absent; replace those logical-OR fallbacks
with nullish coalescing (??) so only null/undefined trigger the fallback — e.g.,
use cookieBannerData?.bannerVisibilityTime ??
cookieBannerData?.bannerInteractiveTime ?? 0 and
cookieBannerData?.viewportCoverage ?? 0 (and mirror the same ?? changes at lines
206-210).
… benchmarks and update pnpm-lock.yaml accordingly. This cleanup enhances project maintainability and reduces unnecessary package bloat.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 3
♻️ Duplicate comments (3)
benchmarks/c15t-react/next-env.d.ts (1)
3-6: Duplicate: Manual import requirestypedRoutesconfiguration.This concern was already raised in a previous review. The manual import of
./.next/types/routes.d.tson line 3 requirestypedRoutes: trueto be enabled innext.config.tsfor Next.js to generate the route types file. Without this configuration, the import will either fail at compile time or be overwritten when Next.js regeneratesnext-env.d.ts.Please ensure
typedRoutes: trueis added to the Next.js config for this benchmark.benchmarks/cookie-control/next-env.d.ts (1)
3-6: Duplicate: Manual import requirestypedRoutesconfiguration.Same issue as in
benchmarks/c15t-react/next-env.d.ts: the manual import on line 3 requirestypedRoutes: trueinnext.config.ts. Verify this is configured for the cookie-control benchmark.benchmarks/onetrust/next-env.d.ts (1)
3-6: Duplicate: Manual import requirestypedRoutesconfiguration.Same issue as other
next-env.d.tsfiles in this PR: the manual import on line 3 requirestypedRoutes: truein the onetrust benchmark'snext.config.ts.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (22)
benchmarks/baseline/package.json(2 hunks)benchmarks/c15t-nextjs/next-env.d.ts(1 hunks)benchmarks/c15t-nextjs/package.json(2 hunks)benchmarks/c15t-react/next-env.d.ts(1 hunks)benchmarks/c15t-react/package.json(2 hunks)benchmarks/cookie-control/next-env.d.ts(1 hunks)benchmarks/cookie-control/package.json(2 hunks)benchmarks/cookie-yes/next-env.d.ts(1 hunks)benchmarks/cookie-yes/package.json(2 hunks)benchmarks/didomi/next-env.d.ts(1 hunks)benchmarks/enzuzo/next-env.d.ts(1 hunks)benchmarks/enzuzo/package.json(1 hunks)benchmarks/iubenda/next-env.d.ts(1 hunks)benchmarks/iubenda/package.json(2 hunks)benchmarks/ketch/next-env.d.ts(1 hunks)benchmarks/onetrust/next-env.d.ts(1 hunks)benchmarks/onetrust/package.json(2 hunks)benchmarks/osano/next-env.d.ts(1 hunks)benchmarks/osano/package.json(1 hunks)benchmarks/usercentrics/next-env.d.ts(1 hunks)benchmarks/usercentrics/package.json(1 hunks)package.json(1 hunks)
🔇 Additional comments (25)
benchmarks/usercentrics/next-env.d.ts (1)
1-6: LGTM! Standard Next.js type declaration file.This file correctly follows Next.js 15 conventions for TypeScript support. The triple-slash directives reference the appropriate Next.js types, and the import statement on line 3 will include the generated route types (created during
next devornext build). This aligns with the PR's broader effort to standardize Next.js configuration across benchmark packages.benchmarks/onetrust/package.json (1)
12-12: Verify Next.js version 16.0.1 exists.Same as
benchmarks/osano/package.json: please verify that Next.js16.0.1is a valid, published version. As of recent releases, Next.js 15.x is the latest stable major version.benchmarks/iubenda/package.json (1)
12-12: Verify Next.js version 16.0.1 exists.Same issue across multiple benchmarks: verify that
16.0.1is a valid Next.js version. Consider using a stable release version like15.xif this version doesn't exist.benchmarks/usercentrics/package.json (1)
12-12: Verify Next.js version 16.0.1 exists.Same version verification issue as other benchmarks: ensure
16.0.1is a valid, published Next.js version.benchmarks/cookie-control/package.json (1)
12-12: Verify Next.js version 16.0.1 exists.Same version verification needed as other benchmarks: confirm
16.0.1is available or use a stable Next.js 15.x version.benchmarks/osano/package.json (1)
12-12: No issues found — Next.js version 16.0.1 is valid and current.Next.js 16.0.1 was published on October 28, 2025, making it the latest patch release and a valid dependency specification. The package.json entry is correct.
Likely an incorrect or invalid review comment.
benchmarks/c15t-nextjs/package.json (2)
2-2: LGTM: Package renamed for consistency.The package name change from "with-c15t-nextjs" to "c15t-nextjs" aligns with the standardized naming convention across benchmark packages in this PR.
26-28: LGTM: Node.js version constraint added.The engines field correctly enforces Node.js >=20.9.0, aligning with other benchmark packages in this refactor.
benchmarks/c15t-react/package.json (2)
2-2: LGTM: Package renamed for consistency.The package name change from "with-c15t-react" to "c15t-react" follows the standardized naming pattern across benchmark packages.
25-27: LGTM: Node.js version constraint correctly specified.The engines field is properly formatted (note the previous "engine" typo was already addressed in commit a823f20) and enforces Node.js >=20.9.0.
package.json (2)
5-12: LGTM: Scripts migrated to new cookiebench CLI.The script commands have been successfully updated to use
pnpm exec cookiebenchinstead of the deprecated benchmark-cli, aligning with the PR's refactoring objectives.
14-26: LGTM: Workspace dependencies updated.The addition of
@consentio/benchmark,@consentio/runner, andcookiebenchas workspace dependencies, along with the removal of the deprecated@cookiebench/cli, properly reflects the new tooling architecture.benchmarks/cookie-yes/package.json (2)
2-2: LGTM: Package renamed for consistency.The package name change from "with-cookie-yes" to "cookie-yes" follows the standardized naming convention across benchmark packages.
24-26: LGTM: Node.js version constraint added.The engines field correctly enforces Node.js >=20.9.0, consistent with other benchmark packages in this refactor.
benchmarks/baseline/package.json (2)
6-6: LGTM: Build script normalized.The trailing space has been removed from the build script, improving consistency.
25-27: LGTM: Node.js version constraint correctly specified.The engines field is properly formatted (the previous "engine" typo was already addressed in commit a823f20) and enforces Node.js >=20.9.0.
benchmarks/iubenda/next-env.d.ts (1)
1-6: LGTM: Standard Next.js TypeScript declarations.This is the standard auto-generated Next.js environment declaration file that provides TypeScript type references for Next.js APIs and routes. The file correctly includes the necessary type references and route imports.
benchmarks/c15t-nextjs/next-env.d.ts (1)
3-3: LGTM: Route types import added.The addition of the routes type import enhances TypeScript support for Next.js routing. This is a standard pattern consistent with other benchmark packages in this PR.
benchmarks/didomi/next-env.d.ts (1)
1-6: LGTM: Standard Next.js TypeScript declarations.This is the standard auto-generated Next.js environment declaration file that provides TypeScript type references for Next.js APIs and routes. The file correctly includes all necessary declarations.
benchmarks/enzuzo/package.json (2)
24-26: Past issue addressed:enginesfield is now correctly spelled.The previous typo of
"engine"has been corrected to"engines"(plural). The field will now be properly recognized by package managers.
11-15: No issues found.Next.js 16 was released October 21, 2025, making the specified version 16.0.1 valid and current. The dependencies are compatible and correctly specified.
benchmarks/osano/next-env.d.ts (1)
1-6: LGTM: Standard Next.js boilerplate.This is the standard Next.js ambient type declaration file. The triple-slash references and route types import are correct and automatically managed by Next.js.
benchmarks/ketch/next-env.d.ts (1)
1-6: LGTM: Standard Next.js boilerplate.This file matches the standard Next.js ambient type declaration format and is correctly configured.
benchmarks/enzuzo/next-env.d.ts (1)
1-6: LGTM: Standard Next.js boilerplate.This ambient type declaration file is consistent with the other benchmark next-env.d.ts files in this PR and follows Next.js standards.
benchmarks/cookie-yes/next-env.d.ts (1)
3-3: No issues found. The typed routes import is part of intentional codebase setup.The import
import "./.next/types/routes.d.ts";appears consistently across all 12 benchmark directories (not just cookie-yes), with identical tsconfig.json configuration including.next/types/**/*.tsin the TypeScript include array. This codebase-wide pattern indicates the import is either auto-generated by Next.js tooling or intentionally added as part of the build process, not a manual violation of the file's warning.
| "scripts": { | ||
| "benchmark": "pnpm exec benchmark-cli benchmark", | ||
| "build": "next build", | ||
| "dev": "next dev --port 3001", |
There was a problem hiding this comment.
Port conflict: multiple benchmarks use port 3001.
This benchmark uses port 3001, which is also used by benchmarks/iubenda/package.json (line 6) and benchmarks/usercentrics/package.json (line 6). Ensure unique ports are assigned across all benchmarks to avoid conflicts during concurrent development.
🤖 Prompt for AI Agents
In benchmarks/cookie-control/package.json around line 6, the dev script uses
port 3001 which conflicts with other benchmarks; update the "dev" script to use
a unique, unused port (for example 3003) by changing the Next.js dev command to
--port <new_port>, and if any local docs, README or CI scripts reference port
3001 for this benchmark, update them to the new port as well.
left a comment
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (6)
package.json (1)
28-30:⚠️ Node version mismatch persists: root allows>=18but benchmarks require>=20.9.0.This conflict was previously flagged (see commit history). All benchmark packages specify
node >=20.9.0, but the root workspace still allows>=18, which could allow developers to install incompatible versions.Apply this diff to align the root constraint with benchmark requirements:
"engines": { - "node": ">=18" + "node": ">=20.9.0" }benchmarks/usercentrics/package.json (1)
6-6:⚠️ Unresolved port conflict from previous review.Line 6 continues to use
--port 3001, which was previously flagged as conflicting with another benchmark (iubenda). Additionally, benchmarks/c15t-nextjs also uses port 3001. This will cause failures if multiple benchmarks are run concurrently. Assign unique ports to each benchmark.packages/cookiebench-cli/package.json (3)
20-30: Critical: Replace alpha dependency with stable release.Line 22 pins
@clack/promptsto1.0.0-alpha.7, which is a pre-release version. This was flagged in a previous review but has not been addressed. Alpha versions can introduce breaking changes and stability issues in production CLI code.The latest stable version is 0.11.0. Upgrade to the stable release.
Apply this diff:
"dependencies": { "@c15t/logger": "^1.0.1", - "@clack/prompts": "1.0.0-alpha.7", + "@clack/prompts": "^0.11.0", "@consentio/benchmark": "workspace:*",
9-11: Add BannerPlugin to inject shebang in CLI entry point.The
rslib.config.tsis missing the BannerPlugin configuration needed to inject#!/usr/bin/env nodeinto the CLI executable. Without this, thedist/index.mjsentry point referenced inpackage.json's bin field will not be executable.Add a plugins array to your rslib configuration:
plugins: [ new BannerPlugin({ banners: ["#!/usr/bin/env node"], }), ]Alternatively, add a postbuild script in
package.jsonto chmod the output file executable.
9-11: Critical: Bin entry requires shebang injection in rslib config.The
rslib.config.tsfile does not include aBannerPluginto inject the required#!/usr/bin/env nodeshebang intodist/index.mjs. Without this, the CLI will not be executable when installed as a global or local bin.Add the following to
packages/cookiebench-cli/rslib.config.ts:import { defineConfig, BannerPlugin } from "@rslib/core"; export default defineConfig({ plugins: [ new BannerPlugin({ banners: { js: ["#!/usr/bin/env node"], }, }), ], // ... rest of config });Alternatively, add a
postbuildscript to chmod the output file:"postbuild": "chmod +x dist/index.mjs".benchmarks/cookie-control/package.json (1)
6-6:⚠️ Port conflict persists: multiple benchmarks use port 3001.The dev script still uses port
3001, which conflicts with other benchmarks in the repository. While this was previously flagged, it remains unresolved. Assign a unique, unused port to avoid conflicts during concurrent development.Apply this diff to resolve the conflict:
- "dev": "next dev --port 3001", + "dev": "next dev --port 3005",Note: Verify that port 3005 (or your chosen port) is not already in use by other benchmarks in the repository.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (17)
benchmarks/baseline/package.json(1 hunks)benchmarks/c15t-nextjs/package.json(2 hunks)benchmarks/c15t-react/package.json(2 hunks)benchmarks/cookie-control/package.json(2 hunks)benchmarks/cookie-yes/package.json(2 hunks)benchmarks/didomi/package.json(2 hunks)benchmarks/enzuzo/package.json(1 hunks)benchmarks/iubenda/package.json(2 hunks)benchmarks/ketch/package.json(2 hunks)benchmarks/onetrust/package.json(2 hunks)benchmarks/osano/package.json(1 hunks)benchmarks/usercentrics/package.json(2 hunks)package.json(1 hunks)packages/benchmark/package.json(1 hunks)packages/cookiebench-cli/package.json(1 hunks)packages/runner/package.json(1 hunks)packages/shared/package.json(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-11-07T16:17:07.589Z
Learnt from: danielsitek
Repo: consentdotio/benchmarks PR: 6
File: benchmarks/with-dgp-cookie-consent/config.json:15-20
Timestamp: 2025-11-07T16:17:07.589Z
Learning: In the cookiebench/benchmark-schema package (packages/benchmark-schema/schema.json), the valid enum values for bundleType are: "esm", "cjs", "iffe", and "bundled". Note that it's "iffe" (not "iife").
Applied to files:
benchmarks/onetrust/package.jsonbenchmarks/cookie-control/package.jsonbenchmarks/enzuzo/package.jsonbenchmarks/iubenda/package.jsonbenchmarks/ketch/package.jsonbenchmarks/c15t-nextjs/package.jsonbenchmarks/cookie-yes/package.jsonpackage.jsonpackages/benchmark/package.jsonbenchmarks/baseline/package.jsonbenchmarks/usercentrics/package.jsonpackages/cookiebench-cli/package.json
🔇 Additional comments (18)
benchmarks/enzuzo/package.json (1)
1-27: LGTM!Manifest is well-structured with correct field naming (
enginesplural), appropriate version constraints, and proper workspace dependency references.benchmarks/osano/package.json (1)
1-27: LGTM!Manifest is consistent with other benchmark packages, with correct field naming and appropriate dependency constraints.
packages/shared/package.json (1)
1-28: LGTM!The manifest properly omits the unnecessary empty
dependenciesobject (addressing prior review feedback) and correctly defines export maps for the ESM module. Build tooling and structure are appropriate for a shared library package.benchmarks/onetrust/package.json (1)
1-27: LGTM!The package updates are clean: name normalized, React and type definitions bumped consistently, unused
cookiebenchdevDependency removed, and benchmark script cleaned up. Theenginesfield is correctly named and set tonode >=20.9.0.packages/runner/package.json (1)
21-27: Verify external dependency version constraints are intentional.The package uses caret constraints on
@playwright/test@^1.57.0andplaywright-performance-metrics@^1.2.4. For@c15t/logger, the exact version1.0.0is pinned. Confirm:
- Whether
playwright-performance-metricsis truly an external package (or should beworkspace:*for internal monorepo dependency)- Whether the exact pinning of
@c15t/logger@1.0.0is intentional (may prevent security updates)benchmarks/iubenda/package.json (1)
1-27: LGTM!Updates follow the established pattern: name normalized, React and type definitions bumped consistently, unused dependencies removed, and
enginesconstraint properly added atnode >=20.9.0.benchmarks/cookie-yes/package.json (1)
1-27: LGTM!Updates are consistent with other benchmark packages: name normalized, React and type definitions bumped uniformly, and
enginesconstraint properly configured atnode >=20.9.0.benchmarks/didomi/package.json (1)
2-27: Consistent dependency and engine constraint updates.Package name, React versions, type definitions, and engines field all follow the PR-wide pattern. Changes are aligned with Next.js 16 and React 19 requirements.
packages/benchmark/package.json (1)
1-38: New benchmark package structure looks sound.The ESM module setup, export surface, and workspace integration align well with the CLI refactoring. Dependencies for performance monitoring (perfume.js, playwright-performance-metrics) and logging are appropriate for a benchmark runner.
benchmarks/ketch/package.json (1)
2-26: Consistent with PR-wide benchmark updates.Name, React versions, type definitions, and engines constraint all follow the established pattern. No issues identified.
benchmarks/c15t-react/package.json (1)
2-26: Consistent update aligned with PR-wide pattern.Name, React versions, types, and engines all properly updated. Changes follow the established benchmark modernization pattern.
benchmarks/usercentrics/package.json (1)
2-26: Otherwise consistent with PR pattern.Name, React versions, types, and engines properly updated. Aside from the port conflict, changes follow the established pattern.
benchmarks/c15t-nextjs/package.json (1)
2-28: Consistent dependency and configuration updates.Name, React versions, types, and engines all properly aligned. Changes follow the PR-wide modernization pattern.
benchmarks/baseline/package.json (1)
6-26: Clean, consistent update across all fields.React versions, type definitions, engines constraint, and scripts all properly updated. Minor formatting improvement (trailing space removed from build script). Changes align with PR-wide modernization.
packages/cookiebench-cli/package.json (2)
1-39: Overall structure and dependencies align with CLI goals.The ESM module setup, export surface, bin entry, and dev tooling are well-configured for a CLI package. Workspace integration with benchmark, runner, and shared packages is appropriate. Aside from the critical issues above (shebang injection and alpha dependency), the manifest is sound.
1-39: The review comment references Next.js 16.0.1 and React 19.2.1 compatibility, butpackages/cookiebench-cli/package.jsondoes not contain either Next.js or React as dependencies. This package is a CLI tool with dependencies like cli-table3, figlet, and picocolors. If compatibility concerns apply, they would be relevant to a different package (possibly one of the workspace packages like@consentio/benchmarkor@consentio/runner). Additionally, the comment contains internal scratchpad notes and malformed verification syntax that should not appear in a review comment.Likely an incorrect or invalid review comment.
benchmarks/cookie-control/package.json (2)
2-2: Approve project rename and engine constraint updates.The project rename to
cookie-controlaligns with the benchmark's purpose. The added Node >=20.9.0 engine constraint is appropriate and consistent with the broader refactor moving to Node 20+. Dependency updates (React 19.2.1, TypeScript types) appear to be routine patch/minor version bumps.Also applies to: 24-26
13-14: React 19.2.1 and react-dom 19.2.1 are compatible with Next.js 16.0.1. Next.js 16 is explicitly designed to support React 19.2, as documented in the official release notes and upgrade guidance. The TypeScript type definitions (@types/react ^19.2.7 and @types/react-dom ^19.2.3) are properly aligned with the React version used, ensuring no type mismatches.
| "scripts": { | ||
| "benchmark": "pnpm exec benchmark-cli benchmark", | ||
| "build": "next build", | ||
| "dev": "next dev --port 3001", |
There was a problem hiding this comment.
Port conflict: dev server uses port 3001.
Line 7 specifies --port 3001, which conflicts with benchmarks/usercentrics/package.json (also port 3001). Running both concurrently in development will cause port binding errors. Assign a unique port to this benchmark.
🤖 Prompt for AI Agents
In benchmarks/c15t-nextjs/package.json around line 7, the dev script sets the
Next dev server to port 3001 which conflicts with
benchmarks/usercentrics/package.json; change the port to a unique unused port
(for example 3002 or another agreed port) by updating the "dev" script to use
--port <unique-port> so both benchmarks can run concurrently without binding
errors.
| "dependencies": { | ||
| "@c15t/logger": "1.0.0", | ||
| "@consentio/shared": "workspace:*", | ||
| "@playwright/test": "^1.57.0", | ||
| "perfume.js": "^9.4.0", | ||
| "playwright-performance-metrics": "^1.2.4" |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Exact version pin on @c15t/logger may limit security patches.
Line 22 pins @c15t/logger to 1.0.0 exactly, which means patch releases with bug fixes and security updates won't be picked up automatically. Consider using ^1.0.0 to allow patch and minor updates, unless a specific version is required for compatibility.
🤖 Prompt for AI Agents
In packages/benchmark/package.json around lines 21 to 26, the dependency for
@c15t/logger is pinned to "1.0.0"; change it to a caret range (e.g., ^1.0.0) to
allow patch/minor updates so security fixes are picked up automatically, then
update the lockfile (run npm/yarn install) and run the test/benchmark suite to
ensure no compatibility break; if a strict pin is required for compatibility,
add a comment in the package.json or repo docs explaining why and note the
process to manually update for security patches.
02620a7 to
83a3559
Compare
83a3559 to
674766e
Compare
left a comment
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (9)
packages/cookiebench-cli/tsconfig.json (1)
1-15: [Duplicate] Missing declaration file generation.This concern was raised in a previous review and remains unaddressed. The
"declaration": trueoption is still missing and would improve consistency with other packages in the monorepo.package.json (1)
28-30: Resolve Node.js engine version mismatch.The root
package.jsonspecifies"node": ">=18", but all benchmark packages require"node": ">=20.9.0". This inconsistency could allow developers installing Node 18 or 19, satisfying the root constraint but failing benchmark requirements. Align the root engines to">=20.9.0"to ensure consistency across the workspace."engines": { - "node": ">=18" + "node": ">=20.9.0" }packages/cookiebench-cli/package.json (1)
20-30: Replace alpha dependency with stable release.
@clack/promptsis pinned to version1.0.0-alpha.7(line 22), which carries stability and breaking-change risks. The latest stable version is 0.11.0. Upgrade to the stable release for production readiness."dependencies": { "@c15t/logger": "^1.0.1", - "@clack/prompts": "1.0.0-alpha.7", + "@clack/prompts": "^0.11.0", "@consentio/benchmark": "workspace:*",benchmarks/iubenda/package.json (1)
6-6:⚠️ Port conflict: 3001 used by multiple benchmarks.This benchmark (and
benchmarks/cookie-control/package.json,benchmarks/usercentrics/package.json,benchmarks/cookie-yes/package.json, andbenchmarks/c15t-nextjs/package.json) all use port3001. Running multiple benchmarks concurrently will cause binding errors. Assign a unique port to this benchmark.packages/benchmark/package.json (1)
22-22: Consider using caret range for@c15t/loggerto allow security patches.Line 22 pins
@c15t/loggerto exact version1.0.0, which blocks automatic security patch updates. All other dependencies use caret ranges (e.g.,^1.0.0). Unless a specific version is required for compatibility, use"@c15t/logger": "^1.0.0"to allow patch and minor releases.benchmarks/cookie-control/package.json (1)
6-6: Port conflict: 3001 already used by multiple benchmarks.benchmarks/usercentrics/package.json (1)
6-6: Port conflict: 3001 already used by multiple benchmarks.benchmarks/cookie-yes/package.json (1)
6-6: Port conflict: 3001 already used by multiple benchmarks.benchmarks/c15t-nextjs/package.json (1)
7-7: Port conflict: 3001 already used by multiple benchmarks.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (18)
benchmarks/baseline/package.json(1 hunks)benchmarks/c15t-nextjs/package.json(2 hunks)benchmarks/c15t-react/package.json(2 hunks)benchmarks/cookie-control/package.json(2 hunks)benchmarks/cookie-yes/package.json(2 hunks)benchmarks/didomi/package.json(2 hunks)benchmarks/enzuzo/package.json(1 hunks)benchmarks/iubenda/package.json(2 hunks)benchmarks/ketch/package.json(2 hunks)benchmarks/onetrust/package.json(2 hunks)benchmarks/osano/package.json(1 hunks)benchmarks/usercentrics/package.json(2 hunks)package.json(1 hunks)packages/benchmark/package.json(1 hunks)packages/cookiebench-cli/package.json(1 hunks)packages/cookiebench-cli/tsconfig.json(1 hunks)packages/runner/package.json(1 hunks)packages/shared/package.json(1 hunks)
🧰 Additional context used
🧠 Learnings (1)
📓 Common learnings
Learnt from: danielsitek
Repo: consentdotio/benchmarks PR: 6
File: benchmarks/with-dgp-cookie-consent/config.json:15-20
Timestamp: 2025-11-07T16:17:07.589Z
Learning: In the cookiebench/benchmark-schema package (packages/benchmark-schema/schema.json), the valid enum values for bundleType are: "esm", "cjs", "iffe", and "bundled". Note that it's "iffe" (not "iife").
🔇 Additional comments (20)
benchmarks/osano/package.json (2)
1-27: Package structure looks good.The manifest is well-structured with proper exports, scripts, and engine constraints. The
"engines"field correctly specifies Node >=20.9.0.
12-14: > Likely an incorrect or invalid review comment.benchmarks/enzuzo/package.json (1)
1-27: Package structure is consistent with similar benchmark packages.The manifest follows the established pattern for benchmark packages. Approval is contingent on the Next.js 16 / React 19.2.1 compatibility verification performed on osano.
benchmarks/onetrust/package.json (1)
13-21: React and type package versions look good.Updates to React 19.2.1 and type packages (24.10.1, 19.2.7, 19.2.3) follow the established pattern across benchmarks and are consistent with the Next.js 16 baseline.
packages/shared/package.json (1)
1-28: Well-structured utilities package manifest.The ESM-focused configuration with clean exports and rslib tooling is appropriate. No runtime dependencies is correct for a shared utilities library.
package.json (1)
5-12: Script refactoring to cookiebench looks good.Delegating benchmark tasks to the unified
cookiebenchCLI simplifies the root workflow. The turbo-based build/check/dev/fmt/lint commands are appropriately maintained.packages/runner/package.json (1)
21-26: Runtime dependency placement of@playwright/testwarrants verification.Typically,
@playwright/testis a dev-only dependency, but here it's listed independencies. Confirm this is intentional (i.e., the runner executes Playwright tests at runtime) rather than a miscategorization.benchmarks/didomi/package.json (3)
2-2: Package renamed fromwith-didomitodidomi.This normalization aligns the package name with the benchmark provider name, improving clarity. The change is safe as this is a private workspace package.
14-15: React dependency updates are safe.Minor version bump from 19.2.0 to 19.2.1 is a patch release with no breaking changes expected.
20-26: Type packages and engine constraint properly updated.Updates to @types packages and the addition of
"engines": {"node": ">=20.9.0"}align with the benchmark modernization strategy across the repository.benchmarks/iubenda/package.json (1)
13-14: React and TypeScript type definitions updated correctly.The updates from React 19.2.0 to 19.2.1 and corresponding TypeScript type definitions are safe patch releases. No breaking changes expected for Next.js 16.
Also applies to: 19-21
benchmarks/cookie-control/package.json (1)
13-14: React and TypeScript type definitions updated correctly.The updates from React 19.2.0 to 19.2.1 and corresponding TypeScript type definitions are safe patch releases.
Also applies to: 19-21
benchmarks/usercentrics/package.json (1)
13-14: React and TypeScript type definitions updated correctly.The updates from React 19.2.0 to 19.2.1 and corresponding TypeScript type definitions are safe patch releases.
Also applies to: 19-21
benchmarks/cookie-yes/package.json (1)
13-14: React and TypeScript type definitions updated correctly.The updates from React 19.2.0 to 19.2.1 and corresponding TypeScript type definitions are safe patch releases.
Also applies to: 19-21
benchmarks/c15t-nextjs/package.json (1)
15-16: React and TypeScript type definitions updated correctly.The updates from React 19.2.0 to 19.2.1 and corresponding TypeScript type definitions are safe patch releases.
Also applies to: 21-23
benchmarks/c15t-react/package.json (1)
14-15: React dependencies, types, and Node engine constraint properly configured.The React 19.2.1 updates, TypeScript type definitions, and correct
enginesfield (plural) are all properly configured. Port 3003 is unique and avoids conflicts with other benchmarks.Also applies to: 20-22, 25-27
benchmarks/ketch/package.json (1)
13-14: React dependencies, types, and Node engine constraint properly configured.The React 19.2.1 updates, TypeScript type definitions, and correct
enginesfield are all properly configured. The dev script uses the default Next.js port 3000 (no explicit port specified), which should not conflict with other benchmarks using 3001 or 3003.Also applies to: 19-21, 24-26
benchmarks/baseline/package.json (3)
25-27: ✅ Critical issue resolved: "engines" field now correctly spelled.The previous critical issue flagged the incorrect field name
"engine"(singular). This has been corrected to"engines"(plural), which ensures npm/Yarn will recognize and enforce the Node.js version constraint.
6-10: Script cleanup aligns with PR objectives.The removal of the
"benchmark"script and whitespace normalization align with the PR's migration to the newcookiebench-clitooling and away from the legacy CLI setup.
14-15: Dependency and type definition updates maintain alignment.React/React-DOM patch bumps (19.2.0 → 19.2.1) and corresponding type definition updates keep runtime and types in sync. This is appropriate for dependency maintenance.
Also applies to: 20-23
eb79b49 to
d84f279
Compare
d84f279 to
674766e
Compare
left a comment
There was a problem hiding this comment.
Actionable comments posted: 4
♻️ Duplicate comments (4)
packages/cookiebench-cli/tsconfig.json (1)
1-15: Past comment: Consider declaration files (already flagged as nitpick).A previous review suggested adding
"declaration": truefor consistency with other packages in the monorepo. While CLIs are typically not consumed as libraries and don't require declaration files, this remains a valid point for internal consistency if utilities might be imported. This is marked as a trivial nitpick and can be addressed separately if desired.benchmarks/cookie-control/package.json (1)
6-6: ** Resolve port conflict: port 3001.**The dev script uses port
3001, which conflicts withbenchmarks/iubenda/package.jsonandbenchmarks/usercentrics/package.json. Assign a unique port (e.g.,3003) to avoid binding failures during concurrent development.packages/benchmark/package.json (1)
21-27: Inconsistent dependency versioning: use^1.0.1for@c15t/loggerinstead of exact pin.Line 22 pins
@c15t/loggerto1.0.1exactly, while other production dependencies use semver ranges (e.g.,^1.57.0for Playwright,^9.4.0for perfume.js). Exact pinning prevents automatic patch and minor updates with security fixes. Use a caret range^1.0.1unless a specific version is required for compatibility."dependencies": { - "@c15t/logger": "1.0.1", + "@c15t/logger": "^1.0.1", "@consentio/shared": "workspace:*",packages/cookiebench-cli/package.json (1)
20-22: Upgrade@clack/promptsto stable release; alpha versions pose stability risks.The
@clack/promptsdependency is pinned to1.0.0-alpha.7(line 22), which remains an alpha/pre-release version. While the version was updated from a previous alpha, the stable release should be preferred to avoid breaking changes and instability in production. A previous review flagged this issue but it was not resolved.Consider upgrading to the latest stable version. If there's a specific reason to use the alpha, document it in a comment or ADR.
- "@clack/prompts": "1.0.0-alpha.7", + "@clack/prompts": "0.11.0",
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (18)
benchmarks/baseline/package.json(1 hunks)benchmarks/c15t-nextjs/package.json(1 hunks)benchmarks/c15t-react/package.json(1 hunks)benchmarks/cookie-control/package.json(1 hunks)benchmarks/cookie-yes/package.json(1 hunks)benchmarks/didomi/package.json(2 hunks)benchmarks/enzuzo/package.json(1 hunks)benchmarks/iubenda/package.json(1 hunks)benchmarks/ketch/package.json(1 hunks)benchmarks/onetrust/package.json(1 hunks)benchmarks/osano/package.json(1 hunks)benchmarks/usercentrics/package.json(1 hunks)package.json(1 hunks)packages/benchmark/package.json(1 hunks)packages/cookiebench-cli/package.json(1 hunks)packages/cookiebench-cli/tsconfig.json(1 hunks)packages/runner/package.json(1 hunks)packages/shared/package.json(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: danielsitek
Repo: consentdotio/benchmarks PR: 6
File: benchmarks/with-dgp-cookie-consent/config.json:15-20
Timestamp: 2025-11-07T16:17:07.589Z
Learning: In the cookiebench/benchmark-schema package (packages/benchmark-schema/schema.json), the valid enum values for bundleType are: "esm", "cjs", "iffe", and "bundled". Note that it's "iffe" (not "iife").
📚 Learning: 2025-11-07T16:17:07.589Z
Learnt from: danielsitek
Repo: consentdotio/benchmarks PR: 6
File: benchmarks/with-dgp-cookie-consent/config.json:15-20
Timestamp: 2025-11-07T16:17:07.589Z
Learning: In the cookiebench/benchmark-schema package (packages/benchmark-schema/schema.json), the valid enum values for bundleType are: "esm", "cjs", "iffe", and "bundled". Note that it's "iffe" (not "iife").
Applied to files:
packages/benchmark/package.jsonpackages/cookiebench-cli/package.jsonbenchmarks/iubenda/package.jsonbenchmarks/cookie-control/package.jsonpackage.jsonbenchmarks/cookie-yes/package.jsonbenchmarks/baseline/package.jsonbenchmarks/c15t-nextjs/package.jsonpackages/cookiebench-cli/tsconfig.json
🔇 Additional comments (12)
packages/cookiebench-cli/tsconfig.json (1)
1-15: Configuration looks solid for a modern CLI project.The TypeScript configuration is well-aligned with current best practices:
- ES2022 target with appropriate libraries (ES2022, DOM, DOM.Iterable)
- Modern module resolution (bundler) and ESNext modules
- Strict type checking enabled with sensible defaults
- Clear directory structure (src → dist)
No issues identified.
benchmarks/enzuzo/package.json (1)
1-27: LGTM!New benchmark package follows the established pattern with correct Node engine constraint and consistent dependency versions across the workspace.
benchmarks/osano/package.json (1)
1-27: LGTM!New Osano benchmark package is well-structured with consistent dependency versions and proper Node engine constraint.
benchmarks/iubenda/package.json (1)
1-27: LGTM!Iubenda benchmark package update follows the established pattern with normalized package name, consistent dependency version upgrades, and proper Node engine constraint.
benchmarks/didomi/package.json (1)
1-28: LGTM!Didomi benchmark package update is consistent with the broader refactor pattern—normalized package name, aligned dependency versions, and proper Node engine constraint.
benchmarks/ketch/package.json (1)
1-27: LGTM!Ketch benchmark package follows the refactor pattern with normalized package name, consistent dependency upgrades, and proper Node engine constraint. Uses default port 3000, avoiding conflicts.
benchmarks/c15t-react/package.json (1)
1-28: ✅ Approved. All dependency updates and engine constraints are correctly configured.The package naming is consistent with the broader refactoring, dependency versions align with the baseline, and the
enginesfield (correctly spelled) enforces Node.js >=20.9.0.benchmarks/cookie-yes/package.json (1)
1-27: ✅ Approved. Package configuration is consistent with baseline updates.Dependency versions, type package updates, and engine constraints are all correctly configured.
benchmarks/c15t-nextjs/package.json (1)
7-7:⚠️ Verify port conflict resolution: dev script still uses port 3001.A previous review flagged that port 3001 conflicts with
benchmarks/usercentrics/package.json. The dev script (line 7) still specifies--port 3001. Confirm whether this conflict has been resolved elsewhere (e.g., usercentrics reassigned to a different port), or assign a unique port to this benchmark.benchmarks/baseline/package.json (1)
1-28: ✅ Approved. Previously flagged critical issues have been resolved.The
enginesfield typo has been corrected (now"engines"not"engine"), and Next.js has been upgraded from 16.0.1 to 16.0.7, addressing the RSC deserialization security vulnerabilities.benchmarks/onetrust/package.json (1)
1-27: ✅ Approved. Previously flagged unused dependency has been removed.The
cookiebenchworkspace dependency that was flagged as unused in the previous review has been removed. Only the schema and config workspace dependencies remain.packages/shared/package.json (1)
1-28: ✅ Approved. Empty dependencies object has been removed.The unnecessary empty
dependenciesfield flagged in the previous review has been removed. The package is appropriately configured as a shared utilities library with no runtime dependencies.
| "private": true, | ||
| "scripts": { | ||
| "build": "next build", | ||
| "dev": "next dev --port 3001", |
There was a problem hiding this comment.
Resolve port conflict: port 3001 is duplicated.
The dev script uses port 3001, which is also used by benchmarks/iubenda/package.json and benchmarks/cookie-control/package.json. Running multiple benchmarks concurrently in development will cause port binding failures.
Assign a unique port (e.g., 3004) or use an environment variable (e.g., ${USERCENTRICS_PORT:-3004}) to allow flexible port assignment.
🤖 Prompt for AI Agents
In benchmarks/usercentrics/package.json around line 6, the "dev" script
hardcodes port 3001 which conflicts with other benchmark packages; update the
script to use a unique default port or an environment variable to allow
overrides (for example change it to use port 3004 or to reference
${USERCENTRICS_PORT:-3004}) so concurrent dev runs won't fail; modify the "dev"
entry accordingly and ensure documentation or run scripts mention the new env
var if used.
| "engines": { | ||
| "node": ">=18" | ||
| } |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
find . -type f -name "package.json" | head -20Repository: consentdotio/benchmarks
Length of output: 726
🏁 Script executed:
#!/bin/bash
# Extract engines constraints from all package.json files
for file in ./package.json ./benchmarks/*/package.json ./packages/*/package.json; do
if [ -f "$file" ]; then
engines=$(jq -r '.engines // empty' "$file" 2>/dev/null)
if [ -n "$engines" ]; then
echo "=== $file ==="
echo "$engines"
fi
fi
doneRepository: consentdotio/benchmarks
Length of output: 932
🏁 Script executed:
cat -n ./package.jsonRepository: consentdotio/benchmarks
Length of output: 1109
🏁 Script executed:
cat -n ./packages/cookiebench-cli/package.jsonRepository: consentdotio/benchmarks
Length of output: 1329
Node.js version mismatch: root constraint incompatible with benchmark packages.
Root package.json specifies "node": ">=18", but all 12 benchmark packages require "node": ">=20.9.0". Developers installing with Node 18 or 19 will fail to meet benchmark requirements, causing runtime errors when executing benchmark scripts (e.g., pnpm benchmark).
Align the root engines constraint to ">=20.9.0" or explicitly document why the root supports a broader range.
🤖 Prompt for AI Agents
In package.json around lines 28 to 30, the root "engines" field lists "node":
">=18" which conflicts with the 12 benchmark packages that require "node":
">=20.9.0"; update the root constraint to ">=20.9.0" (or the same semver range
used by the benchmark packages) so all packages agree, and if you deliberately
want broader support instead, add a clear comment or documentation entry
explaining why the root allows older Node versions and how to run benchmarks
using Node >=20.9.0.
| "dependencies": { | ||
| "@c15t/logger": "1.0.1", | ||
| "@consentio/benchmark": "workspace:*", | ||
| "@consentio/shared": "workspace:*", | ||
| "@playwright/test": "^1.57.0", | ||
| "playwright-performance-metrics": "^1.2.4" | ||
| }, |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Inconsistent dependency versioning: use ^1.0.1 for @c15t/logger instead of exact pin.
Line 22 pins @c15t/logger to 1.0.1 exactly, while other production dependencies use semver ranges (e.g., ^1.57.0 for Playwright). Exact pinning prevents automatic patch and minor updates with security fixes. Use a caret range ^1.0.1 for consistency and to allow security patches.
"dependencies": {
- "@c15t/logger": "1.0.1",
+ "@c15t/logger": "^1.0.1",
"@consentio/benchmark": "workspace:*",🤖 Prompt for AI Agents
In packages/runner/package.json around lines 21 to 27, the dependency
"@c15t/logger" is pinned to "1.0.1" while other deps use semver ranges; change
the version string to "^1.0.1" to allow minor/patch updates and maintain
consistency with the project's dependency versioning; update the package.json
entry, run npm/yarn install to refresh lockfile, and commit the updated
package.json and lockfile.
aad9ccd to
44bef01
Compare
left a comment
There was a problem hiding this comment.
Actionable comments posted: 2
♻️ Duplicate comments (8)
packages/cookiebench-cli/tsconfig.json (1)
8-11: Consider adding declaration file generation for monorepo consistency.This configuration lacks
"declaration": true, which was previously flagged as misaligned with other packages in the monorepo (e.g.,packages/runnergenerates.d.tsfiles). Although CLIs typically don't export public APIs, adding declarations would improve IDE support if other packages import utilities from this CLI.Apply this diff if declaration files are desired:
"strict": true, "skipLibCheck": true, + "declaration": true, "outDir": "dist",packages/cookiebench-cli/package.json (1)
22-22: Replace alpha dependency with stable release.Line 22 pins
@clack/promptsto1.0.0-alpha.7(still an alpha version). According to previous review, a stable release0.11.0is available. Upgrade to the stable version to avoid potential breaking changes and stability issues in production.What is the latest stable version of @clack/prompts npm package?packages/runner/package.json (1)
21-27: Inconsistent exact pin on@c15t/loggerprevents security patches.Line 22 pins
@c15t/loggerto1.0.1exactly, while other dependencies use semver ranges. Exact pinning prevents automatic patch updates with security fixes. Use^1.0.1for consistency and to allow minor/patch updates.benchmarks/usercentrics/package.json (1)
6-6: Port conflict: 3001 is already used by other benchmarks.This was flagged in previous reviews: port
3001is also used bybenchmarks/cookie-control,benchmarks/iubenda, and other benchmarks. Running multiple benchmarks concurrently in development will fail with port binding errors. Assign a unique port (e.g.,3004or use an environment variable like${USERCENTRICS_PORT:-3004}).benchmarks/c15t-nextjs/package.json (1)
7-7: Port conflict: 3001 is already used by multiple benchmarks.This was flagged in previous reviews: port
3001conflicts withbenchmarks/usercentrics,benchmarks/cookie-control,benchmarks/cookie-yes, and others. Change the dev script to use a unique port (e.g.,3002or3005) to enable concurrent development.benchmarks/cookie-control/package.json (1)
6-6: Port conflict: 3001 is already used by multiple benchmarks.This was flagged in previous reviews: port
3001is used by multiple benchmarks includingbenchmarks/iubenda,benchmarks/usercentrics, andbenchmarks/cookie-yes. Running concurrently will cause port binding failures. Assign a unique port (e.g.,3005or higher).benchmarks/cookie-yes/package.json (1)
6-6: Port conflict: 3001 is already used by multiple benchmarks.This was flagged in previous reviews: port
3001is used bybenchmarks/usercentrics,benchmarks/cookie-control,benchmarks/c15t-nextjs, and others. Running multiple benchmarks concurrently in development will fail. Assign a unique port to avoid conflicts.package.json (1)
28-30:⚠️ Node.js version mismatch: root constraint incompatible with benchmark packages.Root
package.jsonspecifies"node": ">=18", but all 12 benchmark packages require"node": ">=20.9.0". Developers installing with Node 18 or 19 will be unable to run benchmarks (e.g.,pnpm benchmark), causing runtime failures.Align the root engines constraint to
"node": ">=20.9.0"to match the benchmark packages' requirement.
📜 Review details
Configuration used: CodeRabbit UI
Review profile: ASSERTIVE
Plan: Pro
Disabled knowledge base sources:
- Linear integration is disabled by default for public repositories
You can enable these sources in your CodeRabbit configuration.
⛔ Files ignored due to path filters (1)
pnpm-lock.yamlis excluded by!**/pnpm-lock.yaml
📒 Files selected for processing (18)
benchmarks/baseline/package.json(1 hunks)benchmarks/c15t-nextjs/package.json(1 hunks)benchmarks/c15t-react/package.json(1 hunks)benchmarks/cookie-control/package.json(1 hunks)benchmarks/cookie-yes/package.json(1 hunks)benchmarks/didomi/package.json(2 hunks)benchmarks/enzuzo/package.json(1 hunks)benchmarks/iubenda/package.json(1 hunks)benchmarks/ketch/package.json(1 hunks)benchmarks/onetrust/package.json(1 hunks)benchmarks/osano/package.json(1 hunks)benchmarks/usercentrics/package.json(1 hunks)package.json(1 hunks)packages/benchmark/package.json(1 hunks)packages/cookiebench-cli/package.json(1 hunks)packages/cookiebench-cli/tsconfig.json(1 hunks)packages/runner/package.json(1 hunks)packages/shared/package.json(1 hunks)
🧰 Additional context used
🧠 Learnings (2)
📓 Common learnings
Learnt from: danielsitek
Repo: consentdotio/benchmarks PR: 6
File: benchmarks/with-dgp-cookie-consent/config.json:15-20
Timestamp: 2025-11-07T16:17:07.589Z
Learning: In the cookiebench/benchmark-schema package (packages/benchmark-schema/schema.json), the valid enum values for bundleType are: "esm", "cjs", "iffe", and "bundled". Note that it's "iffe" (not "iife").
📚 Learning: 2025-11-07T16:17:07.589Z
Learnt from: danielsitek
Repo: consentdotio/benchmarks PR: 6
File: benchmarks/with-dgp-cookie-consent/config.json:15-20
Timestamp: 2025-11-07T16:17:07.589Z
Learning: In the cookiebench/benchmark-schema package (packages/benchmark-schema/schema.json), the valid enum values for bundleType are: "esm", "cjs", "iffe", and "bundled". Note that it's "iffe" (not "iife").
Applied to files:
packages/benchmark/package.jsonpackages/cookiebench-cli/tsconfig.jsonbenchmarks/iubenda/package.jsonpackages/cookiebench-cli/package.jsonpackage.jsonbenchmarks/cookie-yes/package.jsonbenchmarks/c15t-nextjs/package.jsonbenchmarks/baseline/package.jsonbenchmarks/cookie-control/package.json
🔇 Additional comments (19)
packages/cookiebench-cli/tsconfig.json (1)
1-15: Remove unnecessary DOM libraries from CLI configuration.A CLI package typically doesn't require DOM types. The inclusion of
"DOM"and"DOM.Iterable"in thelibarray is unusual unless this package explicitly generates or manipulates DOM-based benchmark code.Verify these are necessary; if not, simplify to just
["ES2022"]."lib": ["ES2022", "DOM", "DOM.Iterable"],Should be:
+ "lib": ["ES2022"],benchmarks/osano/package.json (1)
1-27: Verify bundleType value in related config.json.The related
benchmarks/osano/config.jsonmentioned in the enriched summary usesbundleType: "iife". According to retrieved learnings, valid enum values in the benchmark schema are"esm","cjs","iffe", and"bundled". Note that it's"iffe"(not"iife"). Verify and correct if needed.benchmarks/enzuzo/package.json (1)
1-27: Verify bundleType value in related config.json.Like the osano benchmark, verify that the related
benchmarks/enzuzo/config.jsonusesbundleType: "iffe"(not"iife"). According to retrieved learnings, valid values are"esm","cjs","iffe", and"bundled".packages/shared/package.json (1)
1-28: LGTM!The shared package manifest is well-structured with appropriate exports mapping, scripts, and dev dependencies. No structural issues identified.
benchmarks/iubenda/package.json (1)
1-27: Verify bundleType value in related config.json.The AI summary indicates the related
benchmarks/iubenda/config.jsonincludes bundleType configuration. Verify that it usesbundleType: "iffe"(not"iife"). According to retrieved learnings, valid enum values in the benchmark schema are"esm","cjs","iffe", and"bundled".benchmarks/onetrust/package.json (1)
1-27: Verify bundleType value in related config.json.The AI summary indicates the related
benchmarks/onetrust/config.jsonhas bundleType configuration. Verify that it usesbundleType: "iffe"(not"iife"). According to retrieved learnings, valid enum values in the benchmark schema are"esm","cjs","iffe", and"bundled".benchmarks/ketch/package.json (2)
24-26: ✅ Correct field naming:engines(plural) is now properly used.The critical
"engine"typo from the previous commit has been corrected to"engines". Node.js version constraint>=20.9.0will now be properly enforced by package managers.
12-14: Dependency and type definition updates are consistent.All dependency versions align with the benchmark standardization across the PR (Next 16.0.7, React 19.2.1, type definitions updated to ^19.2.x).
Also applies to: 19-21
benchmarks/c15t-react/package.json (2)
25-27: ✅ Correct field naming:engines(plural) is properly used.The
"engines"field correctly enforces Node.js version>=20.9.0.
6-6: Port assignment avoids conflicts.The dev server port
3003is unique and does not conflict with other benchmarks in this PR.benchmarks/c15t-nextjs/package.json (1)
26-28: ✅ Correct field naming and engine constraint.The
"engines"field correctly specifies Node.js version>=20.9.0.package.json (2)
5-5: ✅ Script migration to cookiebench CLI is correct.The migration from direct benchmark-cli invocations to
pnpm exec cookiebenchcommands aligns with the PR's CLI refactoring objectives. The commands (benchmark,db,results) properly delegate to the new cookiebench workspace package.Also applies to: 8-8, 12-12
14-25: ✅ Dependency updates to support new cookiebench architecture.New workspace dependencies (
@consentio/benchmark,@consentio/runner,cookiebench) and updated tooling versions (biome,turbo,playwright,ultracite,drizzle-kit) are consistent with the PR's refactoring objectives and ecosystem updates.benchmarks/didomi/package.json (2)
25-27: ✅ Correct field naming:engines(plural) is properly used.The
"engines"field correctly specifies Node.js version>=20.9.0.
13-15: Dependency and type definition updates are consistent.All versions align with the benchmark standardization (Next 16.0.7, React 19.2.1, type definitions ^19.2.x).
Also applies to: 20-22
benchmarks/cookie-control/package.json (1)
24-26: ✅ Correct field naming:engines(plural) is properly used.The
"engines"field correctly specifies Node.js version>=20.9.0.benchmarks/cookie-yes/package.json (1)
24-26: ✅ Correct field naming:engines(plural) is properly used.The
"engines"field correctly specifies Node.js version>=20.9.0.benchmarks/baseline/package.json (2)
20-20: Verify @types/node version constraint aligns with engine minimum.The Node.js engine constraint requires
>=20.9.0, but@types/node@^20.10.1only covers>=20.10.1. This creates a gap where developers on Node 20.9.x would lack type definitions. Either adjust the types constraint to^20.9.0or update the engine constraint to>=20.10.1to ensure consistency.Also applies to: 25-26
13-15: Previous critical issues successfully resolved.The prior reviews' critical issues have been properly addressed:
"engines"field is now correctly named (not"engine"), and Next.js has been upgraded from 16.0.1 to 16.0.7 to remediate RSC deserialization vulnerabilities. React/React-dom versions are aligned with Next.js 16.0.7 compatibility requirements.Also applies to: 25-26
| "dependencies": { | ||
| "@c15t/logger": "1.0.1", | ||
| "@consentio/shared": "workspace:*", | ||
| "@playwright/test": "^1.57.0", | ||
| "perfume.js": "^9.4.0", | ||
| "playwright-performance-metrics": "^1.2.4" | ||
| }, |
There was a problem hiding this comment.
🧹 Nitpick | 🔵 Trivial
Inconsistent exact pin on @c15t/logger prevents security patches.
Line 22 pins @c15t/logger to 1.0.1 exactly, while other dependencies use semver ranges (^1.57.0, ^9.4.0). Exact pinning prevents automatic patch updates with security fixes. Consider using ^1.0.1 for consistency.
🤖 Prompt for AI Agents
In packages/benchmark/package.json around lines 21 to 27, the dependency
"@c15t/logger" is pinned exactly to "1.0.1" which blocks automatic patch
updates; change the version string to a semver range (e.g. "^1.0.1") to allow
patch upgrades, then update the lockfile (npm/yarn/pnpm install) and run the
package manager audit/CI to ensure no regressions.
| "bin": { | ||
| "cookiebench": "dist/index.mjs" | ||
| }, |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
# Verify shebang injection configuration for CLI binary
# Check rslib.config.ts for BannerPlugin
echo "=== Checking rslib.config.ts for BannerPlugin ==="
if [ -f packages/cookiebench-cli/rslib.config.ts ]; then
cat packages/cookiebench-cli/rslib.config.ts
else
echo "rslib.config.ts not found"
fi
# Check package.json for build scripts
echo -e "\n=== Checking package.json for build scripts ==="
if [ -f packages/cookiebench-cli/package.json ]; then
cat packages/cookiebench-cli/package.json
else
echo "package.json not found"
fiRepository: consentdotio/benchmarks
Length of output: 1482
Add shebang to CLI binary via build configuration.
The dist/index.mjs output lacks a shebang directive. Add BannerPlugin to packages/cookiebench-cli/rslib.config.ts with stage: "bundle-js" and banner: "#!/usr/bin/env node\n", or implement a postbuild script to inject it after the build completes. Without this, the cookiebench CLI command will not be directly executable.
🤖 Prompt for AI Agents
In packages/cookiebench-cli/package.json around lines 9-11 the published CLI
target dist/index.mjs lacks a shebang so the installed cookiebench binary isn't
executable; fix by updating packages/cookiebench-cli/rslib.config.ts to add a
BannerPlugin configured with stage: "bundle-js" and banner: "#!/usr/bin/env
node\n" so the shebang is emitted during build, or alternatively add a postbuild
script that prepends the shebang to dist/index.mjs and sets the file executable
(chmod +x) after build.
44bef01 to
674766e
Compare
1 participant
Summary by CodeRabbit
New Features
Documentation
Chores
Style
✏️ Tip: You can customize this high-level summary in your review settings.