Repository navigation
Cut large-app build time: fewer inference passes, smaller IR, cheaper split and clang - #585
Merged
Merged
Conversation
… in the candidates
…ad of per-function temp files
…t, same part text
…s instead of one set per SCC
…Returns round, no builtin clones, MIR precision census under MANTICORE_STATS
…uild-speed measurements
…alk; lead with semi-naive scans
…ody whose signature held still wakes only its callees
…tionary, zlib 1.4 changed it (Alpine now ships 1.4)
…tionary, zlib 1.4 changed it (Alpine now ships 1.4)
…ove on every inference
… dependencies first
…ng early left properties narrower than their stores
…changes, and that the module reaches a fixpoint
…ng early left properties narrower than their stores
… opcache and Salsa actually do
…he tables of the inference just before it
…moved no types, a scan can change facts after its walk
…ne name chain re-emitted the receiver per candidate (3M IR lines in one Doctrine method)
… erased spread method calls: one dispatch+unpack body per method name instead of one copy per shape/site
…odule constructor table instead of one inline arm per class; MANTICORE_DYNM_TRACE names the sites that still do not
…tor table too, the owned temp given back after; the test carries a dynamic method call so the table is exercised
…f a switch over every declaration (an Eloquent model calls it several times per attribute; /api/notes 6 to 17 rps)
…r method name instead of ~570 IR lines per site
…ith by-ref holders binding the temp
…per even when a holder has a by-ref param
…stead of a copy per part (part text -21%, objects -21%)
…, clang spends 30% of its CPU on a handful of them
…ter by value on the inline path too
…over 8 MB chunks instead of reading each part whole, free the splitter before clang, write big bodies straight to the staged file (Laravel tree peak 16.3 -> 13.7 GB)
…d of joining them, and release each table once written; staged IR byte-identical
… per distinct name instead of one per row (symfony-demo: 360k strings, 37k distinct, 64 MB of IR)
…a class overloads __get/__set instead of listing every overloading class in each helper (~250 cases in each of ~8 800 helpers on symfony-demo)
# Conflicts: # src/Compile/Mir/Passes/NarrowReturns.php
…and names the constructor C::::construct() in ArgumentCountError (#596)
… typed spread pack takes the shared arm helper (one per pack type; Monomorphize clones forwarders per pack shape, mysqli_result::__mcFetchObject was 13 MB), and an unknown-receiver property read calls the shared per-name reader (symfony-demo closures of 25 MB of __get arms)
… dispatch helper per argument shape instead of splicing every candidate arm at each site; the call-site trace line is passed in, a variable goes over as its slot so a by-reference arm still writes it (MANTICORE_ERINV_TRACE names the sites that stay inline)
…ound by reference corrupts the closure result (#597)
# Conflicts: # src/Compile/Mir/Passes/EmitLlvmCalls.php # src/Compile/Mir/Passes/EmitLlvmObjects.php # src/Compile/Mir/Passes/InferTypes.php # src/Compile/Mir/Passes/Monomorphize.php
… parameter receives it: the caller passes its address, so ref-cell, global and by-ref-param variables share the helper too (symfony-demo: inline dispatch 349 -> 67 MB, staged IR -12%, clang CPU -11%)
…ight instead of hashing straight to a part; an edit still only moves a bucket at a cut (symfony-app, one-function edit: 27/65 parts hit the cache, 15/65 before; symfony-demo largest part 127 -> 67 MB)
… token instead of internal: the property tables name every one, and the split part holding them copied all 8 800 (51 MB on symfony-demo); clang peak per part 3.8 -> 2.4 GB
… cache key instead of a module-wide running number, and a virtual-dispatch thunk numbers its own registers: one edit renumbered every later table, helper and caller, so most split parts missed the object cache
# Conflicts: # tests/aot/cases/stdlib_zlib_inflate_stream.php
…r takes the call site file as well as its line, and other shared helpers no longer push a trace line relative to whichever frame asked first; MANTICORE_EMIT_REVERSE=1 emits functions in reverse to find bodies that depend on emission order
… 64/65 parts from the object cache, 15 before). Shared helpers take their call site line and file as arguments (dispatch arms, the member-call Error, which also stores it in the Error), resolve caches and row/scope tables are named by their key, a virtual-dispatch key spells types instead of interning ids, a key always maps to one name, and emitMixedMagicCall keeps the nodes it wrapped alive: their spl_object_id was handed to later nodes, which then skipped their own __call check
…-scope wrapper for what that scope may reach, instead of a copy of the whole table per scope (symfony-demo: 67 x 4.5 MB of IR)
…plicing is_object/method_exists/get_class at every site: with the wrapped nodes no longer mistaken for one another, every erased fluent call took it (symfony-demo staged IR 2691 -> 2509 MB)
…that builds the argument array itself, instead of an erased __call dispatch and its checks at every site (symfony-demo staged IR 2509 -> 2445 MB, one Configuration builder 3.1 -> 0.9 MB)
…w B][$i] leaked both objects Only for an element read; hoisting a literal out of a write or by-ref chain miscompiled the compiler's own second generation. Fixes #591
…ort on an element: $a['q'] mutated in place changed every copy of $a and every reuse of a literal Through the element's slot, as a & binding takes it, for an outer array whose elements are statically arrays (raw slot, never a reference box); other shapes keep the old path. Fixes #601
…nd magic-only helpers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Compiling a large PHP application spent most of its time re-analysing unchanged code, then handing clang hundreds of megabytes of IR that repeated the same tables and switches. This branch cuts a full cold
-O2build of the Symfony demo from 36 to 14 minutes and of Laravel from 28 to 14, and the heaviest clang part of the demo from 6.8 to 2.4 GB.Results
Full cold build,
-O2, fresh object cache, macOS arm64, one run each on a loaded machine:The demo rows come from these patches merged onto
examples/symfony-demo, with one change from a separate branch (perf/prop-deny-shared: the$o->$namevisibility table, about 280 MB of the IR saving) because that code exists only there. The per-part clang numbers are each part compiled on its own, 8 at a time. The Laravel rows predate the last IR changes.What changed
Inference
array_key_exists,in_array,array_keys,array_values,array_is_list). A clone missed the builtin and scanned the whole array; this was ~35% of the demo build.laravel). A body whose signature held still wakes only its callees; a settled element-as-key body is not re-inferred.MANTICORE_DEPS_SHADOW=1checks that the module ends at a fixpoint.Emit and IR size
$callback(...)on a value of unknown type) runs through one helper per argument shape instead of splicing every candidate arm (function names, callable arrays,__invoke, closures) at each site: 388 to 67 MB of the demo. A variable argument is passed by address, as a&$xparameter receives it, so a by-reference callee still writes the caller's variable; the call site's trace line is passed in.MANTICORE_ERINV_TRACE=1names the sites that stay inline and why.new $class(...$pack)share one helper per method name or one constructor table instead of a copy per site. The erased-receiver member-call Error is one cold helper per method name.__get/__set, and arelinkonce_odrunder the module token, so no split part has to copy all of them.class_uses()resolves the class through the crc32 tree instead of a switch over every declaration.Split and clang
noinline optnone(MANTICORE_OPTNONE_KB, 0 turns it off; not applied at-O3unless set). Clang spent about 30% of its CPU on a handful of such functions.Object cache after an edit
emitMixedMagicCallmarked the calls it wrapped byspl_object_id, an id handed out again once the node was freed: a later, unrelated call could skip its own__callcheck and call a method its class does not have. It keeps the nodes alive now.MANTICORE_EMIT_REVERSE=1emits functions in reverse; diffing the staged module against a normal build lists every body that still depends on emission order.Compiler memory
Yardsticks
MANTICORE_STATS/MANTICORE_PHASE_TRACEprint aprecision:census of erased types in the final MIR.docs/design/inference-fixpoint.mdcovers the measurements and a plan for an inference fixpoint with recorded dependencies (Epic: incremental compilation #498).Generated code
Emit, split and memory changes keep staged IR byte-identical on symfony-app; the IR-size changes are rewrites with the same results per class. The inference changes move staged IR by a few KB out of 745 MB on symfony-app, where one more inference after NarrowReturns moves nothing, as on
main.Bugs found on the way, with repros in
tests/aot/repro/wrong: #595, #596, #597, #598.Test plan
tests/aot/run.sh -ksubsets, no failures:array,string,class,static,prop,foreach,closure,ref_,byref,dyn,generic,reified,narrow,mono,magic,generator,reflection,reflect,split,cache,escape,borrow,spread,erased,overload,property_hook,hook,enum,readonly,invoke,callable,callback,trace,exception,dyn_method,method_exists,array_map,array_walk,usortmagic_shared_default_arm,erased_spread_method_typed_pack,erased_invoke_shared_helperbin/buildself-host, two generations; merged withmainat8c95ce87optnoneon large functions: not measured