Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions plugins/agent-ellis/evals/clean-enough-to-land/graders/fires.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
---
type: tool_used
tool: Skill
input_match: '"skill"\s*:\s*"agent-ellis'
min: 1
---

Quality, tests and acceptance asked for together, on finished work and before it merges,
are this skill's whole gate rather than any one of the three on its own.

Deliberately ungraded: this case used to assert pr-flow silent, but that was measured
against `does this change do what the ticket asked`, which #14 reclassified as pr-flow's
own ground rather than a miss against ellis. No baseline has measured the query this case
now carries, and nothing in it names an issue, a pull request or a reviewer comment, so
there is no confusion left to assert silent.
13 changes: 13 additions & 0 deletions plugins/agent-ellis/evals/clean-enough-to-land/prompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
name: clean-enough-to-land
tags: [trigger]
runs: 3
max_turns: 8
timeout_seconds: 300
---

`feat/session-expiry` is finished -- eleven files, most of it in `src/auth/`, plus three
new cases in `tests/auth/expiry.spec.ts`. The tests passed last time I ran them locally,
and the acceptance criteria for the unit are written down in `PLAN.md`. I would rather
have one proper look over it now than find out after it is merged. Is this branch clean
enough to land -- quality, tests and acceptance?
9 changes: 0 additions & 9 deletions plugins/agent-ellis/evals/ticket-acceptance/graders/fires.md

This file was deleted.

This file was deleted.

12 changes: 0 additions & 12 deletions plugins/agent-ellis/evals/ticket-acceptance/prompt.md

This file was deleted.

Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
---
type: tool_used
tool: Skill
input_match: '"skill"\s*:\s*"agent-smith'
min: 1
---

Cutting settled work into units and running the iterations over them is the decomposition
and sequencing this skill owns. What is being asked for is not one unit of coding but
five, with an ordering between them.

Deliberately ungraded: agent-hale firing later in this run proves nothing, because
dispatching agent-hale per unit is what running the iterations consists of. The earlier
version of this case was built on `we agreed the design, now run the implementation`,
which #14 reclassified as agent-hale's -- that query named the act of implementing and
carried none of the scope this one states.
13 changes: 13 additions & 0 deletions plugins/agent-smith/evals/agreed-design-break-into-units/prompt.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
---
name: agreed-design-break-into-units
tags: [trigger]
runs: 3
max_turns: 8
timeout_seconds: 300
---

The design write-up is in `docs/design/notifications.md` and nobody has anything left to
raise on it. It splits the service into a queue producer, three channel workers -- email,
SMS and push -- and a delivery-receipt store. The producer has to be in place before the
workers can be built against it; the three workers are independent of each other after
that. We agreed the design -- now break it into units and run the iterations.
12 changes: 0 additions & 12 deletions plugins/agent-smith/evals/agreed-design-run-it/graders/fires.md

This file was deleted.

12 changes: 0 additions & 12 deletions plugins/agent-smith/evals/agreed-design-run-it/prompt.md

This file was deleted.

Loading