Repository navigation
Conversation
Contributor
Author
|
@dannycoulombe in this PR I propose to include an initial skill, in an .agents dir (so agent-agnostic), to help with reviews/tests. We could include more related to the dev process. |
From the agent tests behind BenchUp3D#41, the tools agents needed most: - Router profiles on board edges (add_profile, remove_profile), with edges named by the faces they lie between. - Joint trimming and set_joint; get_object shows a joint's geometry, the part actually cut and whether the cut in b is through, stopped or closed. - Finishes (set_finish) and a time estimate (get_estimate). - Boards from a 2D outline with arcs (create_profile_board). - Face operations (split_face, merge_faces), distribute and change_thickness, which keeps outer faces, joint shares and butting parts. - Scene filters, measure_space and find_surfaces for walls and floors made of plain faces, and check_model {only_new} for a batch's new problems. - A PNG preview without a GPU (benchup render, MCP render_view), with joints cut, dimensions and notes, reporting overlapping labels. - Over MCP, run_batch (with dry runs and $N references) and undo. - benchup open on macOS passes the file as a launch argument when BenchUp is not running (the app cannot take an open event at start-up, BenchUp3D#58). The agent guide and CLI docs cover the new tools, MCP and the findings of two agent rounds. The Cutting Plan and the items that need a core decision stay open in BenchUp3D#42; the joint precision limit is BenchUp3D#61. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
crates/benchup-cli/evals holds the 17 cases used to test the tools with agents (12 from BenchUp3D#41 and 5 for the BenchUp3D#42 tools), their seed projects, reference solutions, the prompt every agent gets, a script that prepares a round and the results of rounds 1 to 6. tests/evals.rs replays the reference solutions with the real binary and checks them with check_model, so a change that breaks what agents did fails without running agents. CI now runs the benchup-tools and benchup-cli tests, which it did not run before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
furilo
force-pushed
the
agent-eval-skill
branch
from
November 14, 2026 12:18
d7dd804 to
3daddd7
Compare
Since BenchUp3D#46 the desktop app reloads an open project about a second after its file changes on disk, and asks first when it has unsaved changes. The CLI docs and the agent guide said to run `benchup open` again; they now describe the reload, and keep that advice for BenchUp 0.7.1 and older. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
furilo
force-pushed
the
agent-eval-skill
branch
from
November 14, 2026 12:21
3daddd7 to
55bc85f
Compare
The requests, the prompt and its report sections, the seed project (estanteria.bup is now bookcase.bup, with English part names and no history) and the names in the reference batches are now in English, like the rest of the repository. Part ids are unchanged, so the reference solutions still apply. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
.agents/skills/benchup-agent-eval tells Claude Code or Codex how to run a round of crates/benchup-cli/evals: the regression test first, one subagent per case with its prompt verbatim, an independent check of every result with check_model and a render, and results/round-N.md with the findings sorted into tool bugs, guide gaps, core limits and missing features. .claude/skills links to .agents/skills so Claude Code finds it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
furilo
force-pushed
the
agent-eval-skill
branch
from
November 14, 2026 12:35
55bc85f to
937e871
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Builds on #63: review that one first. Until #63 is merged this PR also shows its two commits; the change here is only the last commit.
Adds a skill that runs a round of the agent evaluation in
crates/benchup-cli/evals, added in #63:.agents/skills/benchup-agent-eval/SKILL.md..agents/skillsis a location shared by agents (Codex reads it), which fits a repository that usesAGENTS.md. The skill tells the agent to:setup.py;check_modeland a render, instead of trusting the reports;results/round-N.md, comparing with the previous round and sorting the findings into tool bugs, guide gaps, core limits and missing features;.claude/skillsis a symlink to../.agents/skills, so Claude Code finds the same skill without a copy.A round costs tokens: the skill says to run one only when asked, and to state how many agents it will start. Rounds 5 and 6 in #63 were run this way.
🤖 Generated with Claude Code