Repository navigation
Verify a compilation before generating its code, never beside it - #138
Merged
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
clang++ -fverify-contractsverified a translation unit on astd::asyncthread (startCppVerifyAsync) while the main thread generated its IR from the sameASTContext(BackendConsumer::HandleTranslationUnit). IR generation is not read-only on the AST: it fills caches of theASTContext(getTypeInfo's map, the linkage cache) and of theSourceManager(its last file and line lookups), which the verifier reads. Concurrent insertion into a map that another thread reads is undefined behavior. A ThreadSanitizerclangcompiling a file with contracts and templates (-g -c) reported races in 4 of 4 runs (found while making #136): CodeGen's debug locations againstVerifier::sourceOfinSourceManager::getLineNumber,setLinkageForGVagainstASTConverter::functionIdentity, andASTContext::getTypeInfoagainstVType::fromQualType. #136's lock orders the verifier's own threads, not the verifier against CodeGen.Change
runCppVerify(CI, Ctx)replacesstartCppVerifyAsyncandfinishCppVerify: it verifies on the calling thread and reports the results through the compiler's diagnostics as before.BackendConsumer::HandleTranslationUnitcalls it beforeGen->HandleTranslationUnit, so nothing else touches the AST meanwhile. Verus runs its verification inside rustc's pipeline the same way, before code generation.A failure is now reported before IR generation, which resets the module once an error is reported (
CodeGeneratorImpl::HandleTranslationUnit), so a compilation that fails verification stops there. Before, IR generation had already released the module, and-emit-llvm -o -printed the IR of a program that failed verification.The overlap bought nothing measurable: optimization and the backend already ran after verification finished, and IR generation itself is small. A whole
-fno-verifycompile ofsuite/llvm_uleb128.cpptakes 0.02 s, against seconds of verification.The IO sandbox
In a build with assertions, LLVM turns on its IO sandbox (
LLVM_ENABLE_IO_SANDBOX, which defaults toLLVM_ENABLE_ASSERTIONS) for the compiler's main thread (driver.cpp,Job.cpp): any file access outside the compilation's file system is a fatal error. The flag is thread-local, so the verifier's own thread never saw it; on the main thread, the first commit of this PR crashed every-fverify-contractscompile withfatal error: error in backend: IO sandbox violation, at the verifier's first read (compute_thread_countreading/proc/cpuinfo). The verifier reads and writes files of its own (the proof cache, the report,-fverify-contracts-out) and runs cvc5 and Lean, sorunCppVerifylifts the sandbox for the whole call (llvm::sys::sandbox::scopedDisable), as the static analyzer does for its reports (ExprEngine.cpp,HTMLDiagnostics.cpp). The compile tests (compile_with_verify.cpp,compile_diagnostics.cpp,compile_options.cpp, ...) are what catch it, in the assertions job. Checked locally with a release build configured with-DLLVM_ENABLE_IO_SANDBOX=ON: the 8 compile-path tests pass; without thescopedDisable,compile_with_verify.cpp,compile_verifies_first.cpp, andcompile_diagnostics.cppfail withfatal error: error in backend: IO sandbox violation.ThreadSanitizer
Checked with a ThreadSanitizer
clang(-std=c++17 -fverify-contracts -g -c, Z3's own reports suppressed) on two probe files of 16 functions each, before this PR (main with #136) and with it, sandbox lifted:The races before were
SourceManager::getLineNumber(CGDebugInfo::setLocationagainstVerifier::sourceOf) andsetCachedLinkage(setLinkageForGVagainstASTConverter::functionIdentity). The probe that verifies is the one that shows the race gone: the failing one never reaches IR generation now.Testing
compile_verifies_first.cpp: a failing function compiled with-emit-llvm -o -reports the failure and nodefine; with-fno-verifythe IR is generated. On main it fails:CHECK-NOT: excluded string found in input,define dso_local noundef i32 @_Z4nexti(...).suite/(81) and the compile-path tests in it (z3_references.cpp,ghost_locals.cpp) pass.Docs
ARCHITECTURE.md ("Compile-Time Integration") says verification runs first and why; release notes ("Running cpp-verify", "Verification before code generation").