Repository navigation
Refute the attribution test through a check any Z3 refutes at once - #139
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The release dry run (S2, run 40818283399) failed one test on Linux:
suite/certified_failure_attribution.cpptook more than its 40 s bound with the archive'scpp-verify, while every Linux CI job passes it in about 2 s. The archive is built in two stages, so its Z3 is built by Clang; CI's Linux release job builds Z3 with GCC.What depends on the build
The test guards one scheduling property: when the complete query of a module is refuted, the failure is reported at once, the obligations only choosing which failure within the complete query's probe (6 million units), instead of waiting for an obligation whose own search does not end (60 s). Its function failed a postcondition whose own query does not end and which the complete query refuted within the probe, but only with a GCC-built Z3:
cpp-verify--jobs=1--jobs=3libz3.aUnresolvedafter 62 sUnresolvedafter 64 sThe relinked binary differs from main's only in Z3's compiler, and its Layer 4 query is byte for byte the same, so the difference is in Z3's search. Over every test program (the Z3 gate's
differential.py, one job per file), the two Z3 builds decided differently only here (lost) and inllvm_uleb128.cpp, where the Clang-built Z3 proveddecode_uleb128_canonical(and soroundtrip_uleb128) and the GCC-built one timed out; total 2501 s and 2492 s. So neither build is worse; this test's input relied on one build's search order, as itsUNSUPPORTED: system-darwin(Apple's Clang builds Z3 there) already said.I tried two scheduling changes before settling on the test. Restarting the complete query in turns of growing length did not help, since a fresh start in the same process mostly repeats the same search (Clang-built Z3,
--jobs=1: 3.8 s, 49 s, and twice over 150 s). Probing with four random seeds instead of one (as F*'s--retrydoes) did not either (--jobs=1: 62 s, 5 of 5). Neither is in this PR.Change
The test's function is now refuted through a check any Z3 refutes at once, after a check whose search ends in no Z3:
solver.timeoutwith either Z3 (20 s timeout: 41 s, both).5:22: error: verification failed: cubes [assertion].UNSUPPORTED: system-darwin.ARCHITECTURE.md's probe paragraph no longer says the probe's stopping point is independent of the build; it says it depends on Z3's own build, with these numbers.