Skip to content

policy: coerce numeric-looking string args for gt/gte/lt/lte constraints #42

Description

@CodewithJha

Problem

Numeric constraint matching uses _as_number, which only accepts int/float (not bool). Numeric strings fail.

Verified:

matches_constraint("210", ArgConstraint(gt=200), context={}) → False
matches_constraint(210, ArgConstraint(gt=200), context={}) → True

Many agent tool UIs / JSON schemas emit "amount": "210" as strings. Policies like sample refund_limit (when: amount gt 200) then miss violations — a correctness bug for the oracle.

Repository evidence

  • packages/mutiny_core/src/mutiny_core/policy/constraints.py_as_number, matches_constraint
  • Sample policy: examples/openai_support_agent/policy.yaml (amount thresholds)
  • Fitness _numeric_boundary may similarly assume numbers (check when fixing)

Why it matters

Deterministic oracle must treat "210" and 210 as the same magnitude for inequality ops, or Mutiny silently under-reports violations on real agents.

Proposed direction

Extend _as_number to parse decimal numeric strings (reject empty, bool, non-numeric). Keep "210usd" failing closed. Document in cheatsheet. Add unit tests.

Acceptance criteria

  • "210" satisfies gt: 200; "150" does not
  • Non-numeric strings still fail numeric ops
  • Bool remains non-numeric (True is not 1)
  • Unit tests in test_policy_evaluator.py / constraints tests
  • No LLM involvement

Testing expectations

uv run pytest tests/unit/test_policy_evaluator.py -q

Scope / NOT included

  • Locale-specific currency parsing beyond simple decimal strings
  • Changing rule kinds

Difficulty

S · core correctness

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workingcoremutiny_core enginehelp wantedExtra attention is neededpriority:P1Important reliability/adoption

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions