Skip to content

fix: paid delegation briefs name spend scope and cap - #270

Merged
lukemaj merged 1 commit into
mainfrom
fix/269-spend-scope
Nov 15, 2026
Merged

lukemaj merged 1 commit into
mainfrom
fix/269-spend-scope

Conversation

@lukemaj

@lukemaj lukemaj commented Nov 15, 2026

Copy link
Copy Markdown
Contributor

What changed: Delegation briefs for paid model or API calls now name the exact calls or cases, the sample count, and a spend cap, and a worker that would exceed the cap stops and reports. The Human Gate line now says paid eval and benchmark runs are credit spending.
Why: A Chromeria worker ran an unrequested full eval suite and drained about 50 USD of OpenAI credits; its brief said only "use the cheapest models" and the existing gate did not stop it.
So what: Review this PR; merge stays with the user per repository policy. No release or publication in this task.

Closes #269.

Elon record

  • Requirements and who asked: user (2026-11-15, Briefs that spend paid credits must name scope and a cap #269): change the rules so an unbounded paid eval run cannot happen again.
  • Deleted: a new global section; a cost-tracking service; project-specific eval tooling (Chromeria owns its runner cap).
  • Bottleneck: the orchestration brief-contents list named no spend scope, and the Human Gate named "credit spending" without saying evals count.
  • Checked myself: read orchestration.md, bounded-delegation.md, global/AGENTS.md and the contract tests; new test failed before the prose edit (4 failures, missing clauses) and the full suite passes after.

Proof

  • New test test_paid_briefs_name_scope_sample_count_and_spend_cap in tests/test_operations_contract.py: red before (FAILED, 4 failures), green after.
  • Full suite: python3 -m unittest discover -s tests → 514 tests, OK.
  • versionctl prepare patch → 17.4.8 to 17.4.9, mirrors and CHANGELOG updated; release-check OK on 96f3fc0.
  • Diff self-reviewed against writing-for-agents and prose guidance; independent review pending.

@lukemaj

lukemaj commented Nov 15, 2026

Copy link
Copy Markdown
Contributor Author

Independent review of 96f3fc0 (relayed from reviewer, codex/gpt-6.1-sol, read-only diff review):

No findings in exact head 96f3fc0.

  • Brief rule (skills/operations/references/orchestration.md:69) meets the scope, sample count, cap, and stop/report requirements.
  • Human Gate (global/AGENTS.md:319) explicitly covers paid evals and benchmarks without broadening authority. Added prose uses named actors, active verbs, adjacent triggers, and no em dashes.
  • Test (tests/test_operations_contract.py:311) asserts exactly the added clauses using existing whitespace-normalization and subtest conventions.
  • 17.4.8 to 17.4.9 is patch-correct; all five configured mirrors and CHANGELOG agree. No unrelated changes or secrets found.

Repository tests were not run by the reviewer; the reported 514-test result remains independently unverified by them. No evals, benchmarks, or paid calls were initiated.

APPROVE: the scoped diff meets the requested content and consistency checks; test execution remains outside this review.

@lukemaj
lukemaj merged commit 23e0674 into main Nov 15, 2026
2 checks passed
@lukemaj
lukemaj deleted the fix/269-spend-scope branch November 15, 2026 20:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Briefs that spend paid credits must name scope and a cap

1 participant