Skip to content

test(base): pin the native U32, Nat and Bool ops to Base's defs - #1630

Open
ScottMoore0 wants to merge 1 commit into
bendlang:mainfrom
ScottMoore0:tests-native-model
Open

ScottMoore0 wants to merge 1 commit into
bendlang:mainfrom
ScottMoore0:tests-native-model

Conversation

@ScottMoore0

Copy link
Copy Markdown
Contributor

The compiler runs U32's arithmetic, bitwise, shift and comparison ops, Nat's ops and U32.to_nat as native C and JS (comp.ts OPERATIONS), not as Base's defs. No test pinned the natives to the defs at edge values. These three tests do:

  • tests/base/u32_native_model.bend: 204 calls of add, sub, mul, div, mod, and, or, xor, min, max, inc, shl, shr, not, shln, shrn, pow and from_nat, at 0, 1, 2^31−1, 2^31, 2^31+1, 2^32−2, 2^32−1, and shifts by 0, 1, 31, 32 and 33.
  • tests/base/u32_native_compare.bend: 40 calls of is_eq, is_ne, is_lt, is_le, is_gt, is_ge and is_zero.
  • tests/base/nat_native_model.bend: 65 calls of Nat's add, sub, mul, div, mod, min, max, double and pow, plus U32.to_nat.

Each has the same shape:

def calls() -> List<U32>:
  [U32.add(4294967295, 1), ...]

def model() -> {calls() == [0, ...] : List<U32>}:
  {==}

def main() -> List<U32>:
  calls()

#|[0, ...]

So the test gate checks that every lane prints the values, and --verdict checks that the values are what Base's defs compute.

What I ran (Linux x86_64 under WSL2, main at e5527d5):

  • All three files: the interpreted, JS and C runs print their #| lines, and --verdict prints ALL PROOFS CHECK (≤1.1 s each).
  • With u32_mul deliberately broken in a copy of comp.ts (C + for *, JS +1): the JS and C lanes of u32_native_model.bend fail. The interpreted lane (which runs Base's defs) and --verdict still pass.
  • The values come from a larger run of 3,774 calls, where the C, JS and interpreted results all agreed and the kernel proved each one by {==} from Base's defs.

Not run: the minis (Metal, CUDA). F32 isn't covered, since Base has no model to compare against.

Found and tested with Claude Code, running Claude Opus 5.5, during an audit of --verdict.

U32's arithmetic, bitwise, shift and comparison ops, Nat's ops and
U32.to_nat run as native C and JS (comp.ts OPERATIONS). Each new test's
main prints a list of calls at edge values (0, 1, 2^31-1, 2^31, 2^32-1,
shifts past 31), and its model def proves by {==} that the same calls
equal the printed values in Base's defs. So the test gate checks every
lane against the values and --verdict checks the values against the
model: a native that drifts from its def fails its lane.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant