[nucliweb]

Summary

Sixth chapter of the MCP server (phase 8a): an eval that asks the same question about a local page to an agent with three setups, and measures what each one costs to get the answer.

• webperf: the MCP server of this repository (node cli/src/bin.js mcp).
• devtools-skill: Chrome DevTools MCP 1.11.0 with the webperf skills, loaded as a temporary plugin; the path the README describes today.
• devtools: Chrome DevTools MCP alone, where the agent measures with traces or its own code.

Each run is claude -p isolated from the settings, skills and MCP servers of the machine (--setting-sources "", --strict-mcp-config), with only the tools of its setup. The prompts read like a person asking and name no tool or snippet. Two fixtures with a known answer, checked first with the server's own tools: an LCP image that the server sends in 2 s (a noise image, so Chrome takes it as a candidate; a flat one is discarded and the LCP falls on the heading), and an "Add to cart" button whose handler blocks the main thread for 250 ms. The grader checks that the report names the cause, and records how the agent measured: published snippets through the server, published scripts given to evaluate_script (as code or by sourcePath), its own code, or other tools.

npm run eval:mcp runs it by hand; it is not part of CI.

Results

Sonnet, 3 runs per case and setup, medians:

Case	Setup	Diagnosis	Input tokens	Turns	Time
LCP	webperf	3/3	12.9k	3	13 s
LCP	devtools-skill	3/3	89.8k	7	30 s
LCP	devtools	3/3	66.0k	5	20 s
INP	webperf	3/3	33.3k	5	15 s
INP	devtools-skill	3/3	104.3k	9	14 s
INP	devtools	3/3	60.7k	6	14 s

• All 18 runs get the diagnosis right: on cases this clear, the setups differ in cost, not in correctness. The MCP server uses 1.8 to 7 times fewer input tokens, fewer turns, and varies least between runs.
• No agent sent a script as code: with Chrome DevTools MCP, the cost comes from the traces, snapshots and code the agent reads and writes. For LCP it used a performance trace and the LCPBreakdown insight; for INP it wrote an Event Timing observer.
• The skills ran in a forked context (context: fork) every time they were invoked (3 times), and the fork answered that it had no page or URL to measure, as a fork does not get the conversation. No published script ran in the 6 runs of that setup. This is a problem of the generated skills, to fix in its own pull request.

Test plan

• Tests first: 11 tests of the grader and the summary (published snippets through the server, published scripts as code and by path, own code, the diagnosis checks, the medians), in test:prompt, 72/72 (local).
• The answer of each fixture with the MCP server: LCP 2036 ms on the img with 98% in resource load time; INP 256 ms on button#add with 251 ms of processing (local).
• A run of 1 per case and setup, then the full run of 3; every transcript of the devtools-skill setup read to confirm what the grader counted. Two runs tried a bash tool and got "No such tool available": the isolation holds (local).
• No browser or server process left by the runs (local).
• CI of this pull request: validate passed in 6m50s on a re-run. The first run failed in third-party-impact.test.js (a domain with 0 ms of main-thread time instead of over 250), a test this pull request does not touch; it looks intermittent and is noted to follow up if it comes back.

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
webperf-snippets	Ready	Preview	Oct 19, 2026 6:03am UTC