Problem
Numeric constraint matching uses _as_number, which only accepts int/float (not bool). Numeric strings fail.
Verified:
matches_constraint("210", ArgConstraint(gt=200), context={}) → False
matches_constraint(210, ArgConstraint(gt=200), context={}) → True
Many agent tool UIs / JSON schemas emit "amount": "210" as strings. Policies like sample refund_limit (when: amount gt 200) then miss violations — a correctness bug for the oracle.
Repository evidence
packages/mutiny_core/src/mutiny_core/policy/constraints.py — _as_number, matches_constraint
- Sample policy:
examples/openai_support_agent/policy.yaml (amount thresholds)
- Fitness
_numeric_boundary may similarly assume numbers (check when fixing)
Why it matters
Deterministic oracle must treat "210" and 210 as the same magnitude for inequality ops, or Mutiny silently under-reports violations on real agents.
Proposed direction
Extend _as_number to parse decimal numeric strings (reject empty, bool, non-numeric). Keep "210usd" failing closed. Document in cheatsheet. Add unit tests.
Acceptance criteria
Testing expectations
uv run pytest tests/unit/test_policy_evaluator.py -q
Scope / NOT included
- Locale-specific currency parsing beyond simple decimal strings
- Changing rule kinds
Difficulty
S · core correctness
Problem
Numeric constraint matching uses
_as_number, which only acceptsint/float(notbool). Numeric strings fail.Verified:
Many agent tool UIs / JSON schemas emit
"amount": "210"as strings. Policies like samplerefund_limit(when: amount gt 200) then miss violations — a correctness bug for the oracle.Repository evidence
packages/mutiny_core/src/mutiny_core/policy/constraints.py—_as_number,matches_constraintexamples/openai_support_agent/policy.yaml(amountthresholds)_numeric_boundarymay similarly assume numbers (check when fixing)Why it matters
Deterministic oracle must treat
"210"and210as the same magnitude for inequality ops, or Mutiny silently under-reports violations on real agents.Proposed direction
Extend
_as_numberto parse decimal numeric strings (reject empty, bool, non-numeric). Keep"210usd"failing closed. Document in cheatsheet. Add unit tests.Acceptance criteria
"210"satisfiesgt: 200;"150"does notTrueis not1)test_policy_evaluator.py/ constraints testsTesting expectations
uv run pytest tests/unit/test_policy_evaluator.py -qScope / NOT included
Difficulty
S · core correctness