[furilo]

Part of #42: the agent tools that agents needed most in the testing behind #41, plus a way to keep checking them with agents.

Tools (benchup-tools, so the CLI, MCP and the in-app assistant share them)

#42 item	Tools
1. Router profiles	add_profile and remove_profile, with edges named by the faces they lie between ([["+z", "-y"]] is the top front edge). get_object lists a board's profiles
3. Trimming, finishes, estimates	trim, keep_end and trim_axis in joint params, plus set_joint; set_finish; get_estimate (finishing, joinery and router time with BenchUp's default allowances, and wood by material)
4. Board from a profile	create_profile_board: a 2D outline with arcs (bulge or sagitta), plane, thickness, grain, material and tag in one call
5. Face operations	split_face (by a line or a plane, pieces ordered and with centres) and merge_faces (coplanar seams, holes such as a mortise's mouth, T-junctions)
6. Distribute	distribute (equal gaps or centres, with or without gaps at the limits)
7. Stock thickness	change_thickness: keeps outer faces, keeps each joint's share of the thickness, and moves boards that only butt against a resized board
8. Scene filters	get_scene ids, depth, fields and sections (the example project goes from 35.8 KB to about 1 KB)
9. Free space and surroundings	measure_space (free gap along each axis to the nearest face) and find_surfaces (walls and floors made of plain faces)
10. Joint geometry	get_object on a joint: region, the tenon or tongue actually cut, depth and direction, and whether the cut in b is through, stopped or closed
11. Rendered preview	benchup render and MCP render_view: a PNG of a standard view without a GPU. Joints and profiles are cut, dimensions and notes are drawn, and overlapping labels are reported
12. check_model against a baseline	check_model {"only_new": true}: only the problems the batch introduced

Also:

• MCP: run_batch (one undo step, dry_run, $N references) and undo.
• CLI: benchup tools --names. --batch always answers with results, and a failed batch leads with its error and hint.
• benchup open on macOS passes the file as a launch argument when BenchUp is not running. The app cannot take an open event while it starts up (macOS: opening a .bup while BenchUp is not running fails with "cannot open files in the BenchUp project format" #58).

Not included:

• the Cutting Plan (item 2), which is being worked on separately;
• the items under Needs a decision in the core. The joint precision limit now has a minimal reproduction in Joinery: second mortise-and-tenon on the same board fails with "Each body must be a closed solid" (precision) #61.

Docs

• docs/agent-guide.md (benchup guide) covers:
  • the new tools;
  • an Over MCP section;
  • what the rabbet joint really cuts, and how to rout a classic rabbet instead;
  • the planes of create_profile_board;
  • the drawer back;
  • the limits for distribute;
  • the precision limit.
• docs/cli.md covers render and the MCP tools.

Agent evaluation (second commit)

crates/benchup-cli/evals keeps the cases used to test the tools with agents that only know what the tools tell them:

• the 12 cases from Add benchup, a CLI and MCP server for editing .bup projects #41 and 5 new ones;
• seed projects and reference solutions;
• the prompt every agent gets;
• setup.py to prepare a round;
• a minimal MCP client;
• the results of rounds 1 to 6.

Summary of the rounds:

Round	Cases	Commands (errors)	Result
4 (before this PR)	12	~245 (~7)	11 ✅, 1 ⚠️
5 (first with these tools)	17	258 (9) for the 12 original cases	15 ✅, 2 ⚠️. Found 9 tool bugs, fixed here
6 (after the fixes, 6 cases)	6	135 (5), down from 159 (9) for the same cases in round 5	5 ✅, 1 ⚠️ partial by design

The cases the tools were made for dropped most: lengthening the desk went from ~61 to 23 commands, and the curved bedside table from 36 to 16, first try.

tests/evals.rs replays the reference solutions with the real binary and checks them with check_model, so a change that breaks what agents did fails without running agents. CI now runs the benchup-tools and benchup-cli tests, which it did not run before. This is the first time the CLI tests run on Windows.

Testing

• cargo test -p benchup-tools -p benchup-cli: 67 tool tests, 22 CLI tests and the evals replay.
• Manual checks with the release binary on the example project: dry-run batches, MCP over stdio and renders of every view.
• benchup-core and benchup-app are unchanged.

🤖 Generated with Claude Code