Repository navigation
[XLS JIT] Improve LLVM IR codegen for partitions, temporaries, literals, and selects - #5164
Closed
copybara-service[bot] wants to merge 1 commit into
Closed
copybara-service[bot] wants to merge 1 commit into
copybara-service[bot] wants to merge 1 commit into
Conversation
…ls, and selects Improve XLS JIT/AOT LLVM IR lowering in `//xls/jit` so that LLVM can inline and discard helper partitions, promote cross-partition temporaries to SSA registers, place constant literal tables in `.rodata`, and lower bit-level selects and boolean trees without control-flow branches or byte-padding masks: 1. **Private partition functions and stack-promoted cross-partition temporaries**: Mark partition functions (`__<fn>_partition_<i>`) with `PrivateLinkage` so LLVM discards standalone partition bodies once inlined into the entry function. Emit typed `inbounds` GEPs in `GetOffsetIntoTempBuffer` (instead of `ptrtoint`/`add`/`inttoptr`) and allocate a local stack `alloca` (`local_temp_buffer`, capped at `kMaxLocalTempBufferBytes = 32 KiB`) when a function has no early-exit/resume points (such as blocking proc receives/sends that require `temp_buffer` state to persist across invocations). This allows LLVM's SROA and dead-store elimination to promote cross-partition temporaries directly to SSA registers. 2. **Read-only module-level constant globals for aggregate literals**: Materialize all `Literal` nodes at their uses (`ShouldMaterializeAtUse`) and emit `private unnamed_addr constant` globals (`CodeModel::Small`) in `NodeIrContext::GetOperandPtr` (as well as on trace/assert string globals, with `Cold` on assertion failure calls) instead of re-initializing stack `alloca` buffers at runtime on every call. 3. **SSA select lowering and `i1` balanced reduction trees**: Lower `Bits`-typed `sel`, `priority_sel`, and `one_hot_sel` directly in SSA registers (`CreateSelect`/`CreateAnd`/`CreateOr`) instead of emitting per-case basic-block branches (`OrInPlace`) or pointer-level `select` + `memcpy`. Truncate 1-bit (`bits[1]`) values in `bit_slice`, `not`, `concat`, n-ary bitwise ops, and `one_hot_sel` to `i1` in SSA registers and reduce `concat`, n-ary ops, and `one_hot_sel` with a balanced binary tree (`TreeReduce`) so expression depth stays $O(\log N)$ within LLVM's `SimplifyDemandedBits` recursion limit (`MaxAnalysisRecursionDepth = 6`). **Runtime and Memory Performance Impact**: - **CPU Runtime**: Strictly faster for JIT and AOT-compiled XLS functions: eliminates per-invocation stack initialization of constant lookup tables, removes per-bit conditional branch mispredictions in `one_hot_sel`, folds 1-bit `one_hot_sel` chains into straight-line bitwise `and`/`or` instructions instead of `cmov`s, and eliminates memory round-trips through `temp_buffer`. - **Memory Usage**: Zero heap overhead; cross-partition temporaries are promoted to registers (bounded by `32 KiB` stack `alloca` prior to SROA), and aggregate literal tables are deduplicated in `.rodata`. PiperOrigin-RevId: 1001458076
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
[XLS JIT] Improve LLVM IR codegen for partitions, temporaries, literals, and selects
Improve XLS JIT/AOT LLVM IR lowering in
//xls/jitso that LLVM can inline and discard helper partitions, promote cross-partition temporaries to SSA registers, place constant literal tables in.rodata, and lower bit-level selects and boolean trees without control-flow branches or byte-padding masks:__<fn>_partition_<i>) withPrivateLinkageso LLVM discards standalone partition bodies once inlined into the entry function. Emit typedinboundsGEPs inGetOffsetIntoTempBuffer(instead ofptrtoint/add/inttoptr) and allocate a local stackalloca(local_temp_buffer, capped atkMaxLocalTempBufferBytes = 32 KiB) when a function has no early-exit/resume points (such as blocking proc receives/sends that requiretemp_bufferstate to persist across invocations). This allows LLVM's SROA and dead-store elimination to promote cross-partition temporaries directly to SSA registers.Literalnodes at their uses (ShouldMaterializeAtUse) and emitprivate unnamed_addr constantglobals (CodeModel::Small) inNodeIrContext::GetOperandPtr(as well as on trace/assert string globals, withColdon assertion failure calls) instead of re-initializing stackallocabuffers at runtime on every call.i1balanced reduction trees: LowerBits-typedsel,priority_sel, andone_hot_seldirectly in SSA registers (CreateSelect/CreateAnd/CreateOr) instead of emitting per-case basic-block branches (OrInPlace) or pointer-levelselect+memcpy. Truncate 1-bit (bits[1]) values inbit_slice,not,concat, n-ary bitwise ops, andone_hot_seltoi1in SSA registers and reduceconcat, n-ary ops, andone_hot_selwith a balanced binary tree (TreeReduce) so expression depth staysSimplifyDemandedBitsrecursion limit (MaxAnalysisRecursionDepth = 6).Runtime and Memory Performance Impact:
one_hot_sel, folds 1-bitone_hot_selchains into straight-line bitwiseand/orinstructions instead ofcmovs, and eliminates memory round-trips throughtemp_buffer.32 KiBstackallocaprior to SROA), and aggregate literal tables are deduplicated in.rodata.