Repository navigation
(feat): compact! — in-place capacity compaction of pool buffers - #45
Merged
Merged
Conversation
Add the per-slot capacity-compaction primitive (CPU) plus the feature design doc. - _slot_capacity / _slot_used / _compact_slot! / _maybe_compact_slot! in src/state.jl. `used` is the max cached wrapper size for the slot (NOT the backing length, which is the high-water mark since _claim_slot! only grows). The backing's Memory is swapped in place for a right-sized one (Vector object identity preserved, so views follow via parent), and the slot's cached wrappers are re-synced. Gated by factor (capacity >= factor*used) and min_bytes. - BitTypedPool / non-Vector backings: no-op fallback (Phase 2+). - test/test_compact.jl: in-place shrink, wrapper re-sync, view follow, both gates, and a function-scoped WeakRef test confirming the old buffer is GC-reclaimed. - docs/design/capacity_compaction.md (implementation plan kept locally under the symlinked docs/plans/).
…able summary
Add the user-facing compact! API that shrinks over-allocated INACTIVE backing
buffers in place (capacity >= factor*used down to ceil(shrink_to*used)), built on
the Phase 1 primitive. Orthogonal to trim!: trim! drops whole inactive slots,
compact! reclaims the retained capacity of the kept ones.
- compact!(pool; factor=10, shrink_to=1.5, min_bytes=2^20, force_gc=false)
- compact!(pool, ::Type{T}; kwargs...) — guarded, never creates a fallback pool
- compact!(; kwargs...) — task-local pool
- compact!(::DisabledPool[, ::Type{T}]) — zero-summary no-ops
- Returns (; slots_compacted, bytes_reclaimed, gc_triggered)
Type stability mirrors trim!: @generated fixed-slot fold + ::NTuple{2,Int}-asserted
others loop; verified via Base.return_types/@inferred (concrete summary).
Legacy (<1.12): warn-once no-op forms + literal _ZERO_COMPACT_SUMMARY, required now
because exporting compact! obligates a definition on the legacy path (Aqua).
Export compact!. Tests: Tier-1 inactive reclaim, active-untouched-by-default,
per-type guard, type stability, no-op/force_gc, DisabledPool + task-local routing,
and self-heal (re-acquire after compaction, reuse and regrow).
…y parity Complete the CPU compact! surface. active=true (Tier 2) extends the slot scan from inactive-only (n_active+1:end) to all slots (1:end), so buffers the caller is STILL HOLDING are compacted — the feature's reason for existing over trim!. Safe because the Phase 1 primitive already re-syncs every cached wrapper's :ref and keeps the backing Vector's identity (held wrappers and their views follow the in-place Memory swap), and target >= used never drops live elements. - active::Bool=false threaded through _compact_counts!/_compact_fixed_counts!(@generated)/ _compact_others_counts!/_compact_one_counts! and every public form. - compact!(pool, types::Type...): @generated varargs mirroring trim! (single summary, type-stable — @inferred + return_types == Summary). - DisabledPool + legacy (<1.12): active kwarg on all forms + varargs no-ops. Tests: active slot held by user compacted in place (data intact, wrapper + view of the held wrapper both read and write through the swap), active=false leaves active alone, active=true also sweeps inactive, varargs multi-type, varargs honors active+force_gc, DisabledPool varargs no-op.
…extent
compact!(pool; active=true) could shrink a slot's backing below a live view held by
the caller, causing out-of-bounds access. Root cause: _slot_used computed the live
extent by scanning the cached Array wrappers (arr_wrappers), but acquire_view! returns
a fresh, UNCACHED SubArray/ReshapedArray that records no wrapper. A slot carrying a
small cached wrapper plus a larger live view under-reported its extent, so active
compaction shrank the backing below the view.
Fix: record each slot's current logical extent in a new TypedPool field slot_extents,
set in _claim_slot! (the shared primitive for BOTH the acquire! and acquire_view!
paths). _slot_used now reads slot_extents[slot] — view-aware and path-independent, so
target = max(used, ceil(shrink_to*used)) always covers every live reference.
slot_extents is kept parallel to tp.vectors (push in _claim_slot!, resize in
_trim_counts! via _trim_slot_extents!, clear in empty!). Backing-length semantics
(high-water mark) are unchanged.
Bonus: _slot_used now returns a concrete Int (Vector{Int} read) instead of Any
(length(::Any) over the Union-typed wrapper cache), so compact!(pool)/compact!(pool,T)
drop from ~224 B / 32 B to 0 B on the cold non-bloated scan — matching trim!.
Tests: regression testsets for a live 1-D SubArray and N-D ReshapedArray extent
(length-only assertions, no OOB indexing). Verified: bug repro now safe (backing 1500
covers view 1000, data intact), 0-alloc cold scan, _slot_used infers Int64.
Bring compact! to the GPU backends, mirroring the CPU surface. Each backend gets:
- slot_extents::Vector{Int} on its TypedPool (CuTypedPool / MetalTypedPool), recorded
in _{cuda,metal}_claim_slot! and cleared/trimmed parallel to vectors — same model as
the CPU bugfix, so _slot_used is view-aware and allocation-free (concrete Int).
- _slot_capacity (device buffer: maxsize ÷ aligned_sizeof for CUDA, ÷ sizeof for Metal),
_slot_used, _maybe_compact_slot!, and an in-place _compact_slot! that allocates a
smaller device buffer, copies live elements, swaps the backing's DataRef (keeping
object identity), and re-syncs cached wrappers via the existing refcount helper
(_update_{cuda,metal}_wrapper_data!), freeing the old device buffer.
- public compact!(pool) / (pool,T) / (pool,T...) mirroring trim!: foreach_fixed_slot
with box-free Ref accumulators (backend-specific fixed-slot set), reusing the generic
_compact_counts!/_compact_others_counts!/_compact_summary. CUDA force_gc also runs
CUDA.reclaim() like trim!.
Note: GPU has NO uncached views (acquire_view! == acquire! → cached wrapper), so the
CPU view-extent bug is structurally absent here; slot_extents is still used for a
uniform, type-stable _slot_used.
Tests: test/{metal,cuda}/test_compact.jl (inactive shrink, active=true on a held slot
with device data intact + writable, no-op/force_gc, type stability, per-type, varargs).
Metal verified on Apple Silicon hardware (full Metal suite green). CUDA mirrors the
working Metal path and is syntax-verified; runtime verification requires NVIDIA CI.
…default Change the compact! default from active=false to active=true across all forms (CPU, legacy, Metal, CUDA, DisabledPool). compact!'s purpose is to reclaim capacity from arrays still in use, so the no-arg compact!(pool) now does that by default; active=false is the explicit opt-out (inactive slots only). This is safe-by-construction: the factor gate (capacity >= 10*used by default) means a normally-sized live array is never touched — only grossly over-allocated ones, which is exactly what a deliberate compact! call wants reclaimed. The in-place swap re-syncs held wrappers/views, so live references follow. Tests: rewrote the CPU 'default leaves active untouched' testset into 'default compacts active bloat; active=false opts out' + a factor-gate test proving a normally-sized live array is left alone. Docstring updated (active=true default; active=false opt-out). Full CPU+Metal suite green (Metal active tests already use explicit active=).
…e kwargs _maybe_compact_slot! computed target = ceil(Int, shrink_to*used) before checking it against capacity; an absurd shrink_to (e.g. 1e300) overflowed Int → InexactError. Clamp the shrink goal to [used, cap] by capping the Float at cap before ceil: target = max(used, ceil(Int, min(Float64(cap), shrink_to*used))). This is also semantically correct — shrinking to >= capacity is meaningless and now a clean no-op (reclaim 0 → skipped). Applied identically to CPU, Metal, CUDA. Robustness sweep (24 calling contexts incl. empty pool, global scope, Bit slots, mixed types, @with_pool, repeated, self-heal, zero-length, DisabledPool, task-local, and pathological factor/shrink_to/min_bytes) now passes with 0 errors. Test added: extreme shrink_to is a safe no-op, not a throw.
There was a problem hiding this comment.
Pull request overview
This PR introduces compact!, a new manual reclamation operation that shrinks over-allocated capacity of pool backing buffers in place (preserving wrapper/view identity), enabling memory reclamation even for slots that are still active/in use. It extends the pool state to track per-slot logical extents and implements CPU + Metal + CUDA parity, with a legacy (Julia < 1.12) warn-once no-op for API stability.
Changes:
- Add
compact!API (pool-wide, per-type, varargs, and task-local) plusDisabledPool/legacy no-op implementations. - Track per-slot current logical extent via
slot_extentsacross claim paths, and use it to safely compact active and inactive slots. - Add comprehensive CPU/Metal/CUDA tests and wire them into the test runners.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 4 comments.
Show a summary per file
| File | Description |
|---|---|
src/AdaptiveArrayPools.jl |
Exports compact! and wires it into the public API surface. |
src/types.jl |
Adds slot_extents to TypedPool to track per-slot logical extents. |
src/acquire.jl |
Records slot_extents on slot claims (including placeholder/reshape path). |
src/state.jl |
Implements the compact! primitives, summaries, and public overloads (CPU path). |
src/legacy/state.jl |
Adds legacy (Julia < 1.12) compact! warn-once no-op returning a zero summary. |
ext/AdaptiveArrayPoolsMetalExt/types.jl |
Adds slot_extents to Metal typed pools for compaction parity. |
ext/AdaptiveArrayPoolsMetalExt/acquire.jl |
Records slot_extents on Metal claim paths. |
ext/AdaptiveArrayPoolsMetalExt/state.jl |
Implements Metal compact! including in-place device buffer swap and wrapper re-sync. |
ext/AdaptiveArrayPoolsCUDAExt/types.jl |
Adds slot_extents to CUDA typed pools for compaction parity. |
ext/AdaptiveArrayPoolsCUDAExt/acquire.jl |
Records slot_extents on CUDA claim paths. |
ext/AdaptiveArrayPoolsCUDAExt/state.jl |
Implements CUDA compact! including in-place device buffer swap, wrapper re-sync, and optional CUDA.reclaim(). |
test/test_compact.jl |
Adds CPU unit/regression tests covering active/inactive compaction, gates, GC reclaim, and type stability. |
test/runtests.jl |
Includes the new CPU compact! test file in the main test suite. |
test/metal/test_compact.jl |
Adds Metal compact! tests (inactive + held active compaction, type stability, DisabledPool). |
test/metal/runtests.jl |
Includes Metal compact! tests in the Metal suite. |
test/cuda/test_compact.jl |
Adds CUDA compact! tests (inactive + held active compaction, type stability, DisabledPool). |
test/cuda/runtests.jl |
Includes CUDA compact! tests in the CUDA suite. |
docs/design/capacity_compaction.md |
Adds design documentation for capacity compaction / compact!. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #45 +/- ##
==========================================
+ Coverage 95.63% 95.73% +0.09%
==========================================
Files 14 14
Lines 3391 3493 +102
==========================================
+ Hits 3243 3344 +101
- Misses 148 149 +1
🚀 New features to boost your workflow:
|
All four are documentation-only consistency fixes after the active=true default flip and the slot_extents mechanism: - _compact_counts! comment: active=true is the default (was labeled active=false). - compact! docstring: current logical size lives in slot_extents (set by the shared claim path, covering acquire! and acquire_view!), not 'the cached wrapper'. - design doc status: Proposed → Implemented (CPU+Metal+CUDA, 1.12+). - design doc API: default active=true (was false); 'Proposed API' → 'API (as implemented)'.
Codecov showed the entire legacy compact! no-op block (src/legacy/state.jl, the warn-once shims for pool / pool,T / varargs / no-arg / DisabledPool forms) as uncovered: the <1.12 test branch had test_trim_legacy.jl but no compact equivalent, so those lines were never executed on the LTS CI job. - Add test/test_compact_legacy.jl (mirrors test_trim_legacy.jl) exercising every legacy compact! form incl. the Type... varargs and active/force_gc kwargs; wire it into the <1.12 branch of runtests.jl. - Also add the missing trim! varargs assertions to test_trim_legacy.jl (the legacy trim!(pool, T...) / trim!(::DisabledPool, T...) lines were uncovered too). Verified on Julia 1.10 LTS: full suite green, legacy/state.jl coverage up to ~95-96%.
mgyoo86
added a commit
that referenced
this pull request
Jul 1, 2026
* refactor: rename auto_compact → auto_manage (unify the maintenance engine)
The background-maintenance engine is about to gain a second action (auto-trim), so
"compact" undersells it. Rename the unreleased umbrella `auto_compact` → `auto_manage`
across CPU, legacy, Metal, CUDA, and tests — const `AUTO_COMPACT`→`AUTO_MANAGE`,
Preference `auto_compact`→`auto_manage`, `enable_/disable_/…_enabled`, the engine
internals (`_run_/_maybe_/register_/_…_sweep!`, `_AUTO_MANAGE_{REGISTRY,TIMER,CONFIG,LOCK}`,
`_AutoManageConfig`), and `src/auto_compact.jl`→`src/auto_manage.jl` (+ legacy + exts +
test files). Pure rename — no behavior change.
The compact *action* is untouched: `compact!`, the `_compact_requested` flag, and
"compaction" wording all stay. Safe to rename: no released/registered version ships
`auto_compact` (latest tag v0.3.7 predates #44/#45/#46; version still 0.3.7).
Validated: renamed CPU + legacy tests green; full Metal suite green (ext rename loads);
runic clean; 0 leftover `auto_compact`/`AUTO_COMPACT`/`AutoCompact` references. CUDA
rename validated on the remote box.
* feat(auto_manage): auto-trim — periodic working-set tail reclamation (CPU)
Second action of the auto_manage engine: every `trim_interval`, trim each type's
slot tail down to its recent working-set peak, reclaiming the inactive tail that
`compact!` only shrinks. Catches the common case of a pool that peaked wide then
runs narrow (still entered, so whole-pool-idle schemes never reclaim it).
- `_ac_peak_n_active::Int` on TypedPool + BitTypedPool: peak `n_active` since the
last trim. One branchless `max` in the claim paths (`_claim_slot!`, `get_bitarray!`),
gated by the compile-time `AUTO_MANAGE` const → DCE'd to zero when off (measured
~0.15 ns/acquire, zero-alloc when on). Reset to 0 at each auto-trim, so it doubles
as the period accumulator (trim targets the *recent* peak, never the all-time max).
- `@atomic _trim_requested::Bool` on AdaptiveArrayPool (+ both ctor paths), parallel
to `_compact_requested`. The Timer sets it every `trim_every_ticks =
round(trim_interval/interval)` sweeps via a global tick counter.
- `_trim_to!(tp, keep)`: generalizes `trim!` (truncate to `keep`, clamped ≥ n_active).
- `_run_auto_manage!`: services `_trim_requested` (→ `_auto_trim!`, trim each type to
its peak then reset) then `_compact_requested` (→ compact!), each guarded by its flag.
The entry hook is unchanged (still triggers on `_compact_requested`, set on the same
sweep). `hasfield` guards keep it backend-safe before GPU parity lands.
- `enable_auto_manage!(; trim_interval = 120.0)`; `Inf` disables auto-trim.
Tests (test/test_auto_trim.jl): peak tracking, `_trim_to!`, auto-trim to the recent
peak / to 0 for an unused type, no-op when unflagged, timer cadence, `Inf` disable,
zero-alloc hot path, and a RUNTIME_CHECK=1 regression — auto-trim drops the slot
reference without swapping the buffer, so an escaped view's poison survives (unlike
the compact! poison-undo bug fixed in 988c50f). Tier-1 green.
* feat(auto_manage): auto-trim Metal parity
Mirror the CPU auto-trim data layer (commit 24586d9) onto the Metal backend. The
service layer (`_trim_to!`, `_auto_trim!`, `_run_auto_manage!`) is generic over the
abstract pool supertypes and already worked for Metal, so this is purely the two
telemetry fields plus the claim-path `max`:
- `_ac_peak_n_active::Int` on MetalTypedPool: peak `n_active` since the last trim.
One branchless `max` in both `_metal_claim_slot!` methods, gated by the compile-time
AUTO_MANAGE const -> DCE'd to nothing when off (one Int write when on).
- `@atomic _trim_requested::Bool` on MetalAdaptiveArrayPool (+ ctor), parallel to
`_compact_requested`. The base-module Timer sweep's `hasfield` guard now const-folds
true for Metal pools, so the sweep flags trim every `trim_interval`; the unchanged
entry hook (fires on `_compact_requested`, set on the same sweep) lets the generic
`_run_auto_manage!` service both at the `@with_pool :metal` safepoint.
Tests (test/metal/test_auto_trim.jl): the CPU suite mirrored for Metal -- peak
tracking through `_metal_claim_slot!`, `_trim_to!` truncation + regrow, auto-trim to
the recent peak / to 0 for an unused type, no-op when unflagged, an R=1 no-buffer-swap
guard (the dropped slot's MtlArray DataRef is left intact, unlike compact!), and the
sweep flagging `_trim_requested` on a registered Metal pool. Full Metal suite green
locally (57s).
* feat(auto_manage): auto-trim CUDA parity
Mirror the CPU/Metal auto-trim data layer onto the CUDA backend (commits 24586d9,
ab9be38). The service layer (`_trim_to!`, `_auto_trim!`, `_run_auto_manage!`) is
generic over the abstract pool supertypes and already worked for CUDA, so this is
purely the two telemetry fields plus the claim-path `max`:
- `_ac_peak_n_active::Int` on CuTypedPool: peak `n_active` since the last trim. One
branchless `max` in both `_cuda_claim_slot!` methods, gated by the compile-time
AUTO_MANAGE const -> DCE'd to nothing when off. The operands are host-side `Int`
fields (NOT device arrays), so this is a CPU integer op, not a GPU scalar op.
- `@atomic _trim_requested::Bool` on CuAdaptiveArrayPool (+ both the main ctor and
the `_transfer_cuda_pool` migration ctor, which resets it like `_compact_requested`),
parallel to `_compact_requested`. The base-module Timer sweep's `hasfield` guard now
const-folds true for CUDA pools, so the sweep flags trim every `trim_interval`; the
unchanged entry hook (fires on `_compact_requested`, set on the same sweep) lets the
generic `_run_auto_manage!` service both at the `@with_pool :cuda` safepoint.
Tests (test/cuda/test_auto_trim.jl): the CPU/Metal suite mirrored for CUDA -- peak
tracking through `_cuda_claim_slot!`, `_trim_to!` truncation + regrow, auto-trim to
the recent peak / to 0 for an unused type, no-op when unflagged, an S=1 no-buffer-swap
guard (the dropped slot's CuArray DataRef is left intact, unlike compact!), and the
sweep flagging `_trim_requested` on a registered CUDA pool. Validated locally without a
GPU via extension precompile + struct/ctor-arity reflection (fieldcount 7 / 18); the
device-level suite runs on the CUDA box.
* test(auto_manage): end-to-end integrity + real-time cadence
Close the gap between the unit suites (which test the mechanism in isolation -- set a
flag, call the service; the depth-1 safepoint no-op) and the safety guarantee that
actually matters: the real background Timer firing repeatedly mid-computation must
never corrupt a result.
test/test_auto_integration.jl:
- A. Differential integrity: a deterministic, pool-heavy simulation (varying array
size -> compact bloat; varying concurrency width -> trim tail; nested depth-2 scope)
run with auto-manage OFF then ON (real Timer at 0.01s, ~40 sweeps over the run).
Asserts the two result vectors are BIT-IDENTICAL -- auto-compact/auto-trim reclaim
memory between scopes, never change a computed value. Plus self-heal-after-reclaim
correctness, and measurable reclamation (slot tail + byte bloat both shrink).
- B. Real-time cadence: over real elapsed seconds, asserts the Timer flags compaction
every tick and auto-trim only every K-th tick (K = round(trim_interval/interval)),
observed without servicing so the raw cadence is visible.
Timing margins match/exceed the existing sleep-based timer tests; verified stable
across repeated local runs (9/9 testset, ~5s). CPU only for now; Metal/CUDA mirrors
to follow.
* test(auto_manage): GPU integrity + real-time cadence (Metal + CUDA)
Mirror the CPU end-to-end integration suite (006552c) onto Metal and CUDA. Same two
guarantees -- (A) the real Timer firing repeatedly mid-computation produces BIT-IDENTICAL
results vs auto-manage off, and (B) the compact-every-tick / trim-every-K-th cadence over
real elapsed seconds -- adapted for GPU:
- No element-wise writes (that is GPU scalar indexing): each slot is fill!ed with a small
integer-valued Float32 kept under 2^24 * value, so the on-device `sum` is an exact integer
independent of the parallel reduction order (bit-reproducible run-to-run, verified),
while fill! over the whole buffer keeps the test sensitive to any aliasing/corruption.
Metal: full suite green locally (integration testset 9/9, 4.1s). CUDA: identical mirror,
parse-checked locally; the device suite runs on the GPU box.
* refactor: rename _ac_peak_n_active → _am_peak_n_active (auto_manage namespace)
The `_ac_` prefix was a leftover from the pre-rename `auto_compact` era; the field is
auto-manage telemetry (consumed by auto-trim), so `_am_` matches AUTO_MANAGE and the
`auto_manage` umbrella naming used everywhere else. Pure mechanical rename of the private
field across CPU + Metal + CUDA + tests — no public API, branch unreleased. CPU auto
suites green; Metal full suite validating.
* fix(compact): active compact! no longer strands reshape! aliases
reshape!(pool, A, dims) with ndims(A) != length(dims) stores its reshaped wrapper in a
SEPARATE placeholder slot while aliasing A's backing memory. _compact_slot! re-pointed
only the wrappers cached AT the compacted slot index, so the cross-slot reshape alias was
left on the freed backing after an active=true compact! -- A followed the new memory while
the reshape view silently diverged (reproduced with default compact! settings on a 1.6 MB
slot, not just contrived parameters).
Fix (all three backends): capture the slot's backing identity BEFORE the swap and re-point
EVERY cached wrapper that shares it, across all slots -- not just wrappers[slot]:
- CPU: match by Memory identity (getfield(w,:ref).mem === old_mem), setfield!(:ref).
- Metal/CUDA: match by DataRef refcount identity (getfield(w,:data).rc === old_rc) and
migrate via the refcount-balanced _update_*_wrapper_data! helper -- the same .data.rc
identity reshape! itself uses to decide buffer sharing.
The scan runs in the cold compaction path (which already allocates a backing + copyto!s),
so the extra work is negligible; every pool wrapper points at offset 0 of its backing, so
Memory/DataRef identity is a precise, offset-safe alias key.
TDD: RED test pinned per backend (test/{,metal/,cuda/}test_compact.jl) -- write through one
view, read the other, plus a shared-backing assertion -- failed before, passes after. CPU
full suite + Metal full suite green; CUDA precompiled + parse-checked (device suite remote).
* docs: document auto_manage (Automatic Memory Management page + API/config)
Add a Features page covering the background auto-compact/auto-trim engine — what it
reclaims, the @with_pool safepoint model (why it can't change results), default-on, the
enable/disable knobs, the zero-alloc trade-off + auto_manage=false escape hatch, and
manual compact!/trim!. Add compact! + the auto_manage symbols to the API reference
tables and an auto_manage preference section to Configuration. Also fix a pre-existing
broken [Safety](safety.md) link in configuration.md (→ ../safety/overview.md).
* fix(auto_manage): address PR #47 review + raise patch coverage to 100%
Review fixes (GitHub Copilot):
- Legacy (Julia <1.12) `enable_auto_manage!` shim now accepts (and ignores) the new
`trim_interval` keyword, so the public API is call-compatible across all supported Julia
(previously a MethodError on old Julia when passing trim_interval).
- Refresh the `enable_auto_manage!` docstring: show `trim_interval`, document auto-trim,
generalize the "what gets managed" note (compaction + trim), drop the typo.
- Fix rename artifacts from the auto_compact->auto_manage token replace: "auto-manageion"
-> "auto-management" / "auto-manageed" -> "auto-managed" (15 comments across 10 files),
and the misleading "Auto-Compact" file/section headers -> "Auto-Manage".
Patch coverage (was 94.07% on src/auto_manage.jl, the only file below 100%):
- Cover the `_auto_trim!` `others`-type loop with a non-fixed-slot (UInt8) auto-trim test.
- Cover the two `_run_auto_manage!` try/catch+@warn defensive branches by parking a non-pool
object in both others collections (values(others) for trim, _others_values for CPU compact)
— mirrors the existing throwing-sweep test. src/auto_manage.jl now 100% (90/90).
* (release): bump version to v0.3.8
* feat(auto_manage): descriptive config kwargs + LocalPreferences defaults
Polish the public tuning API before it ships in v0.3.8:
- Rename `enable_auto_manage!` keywords to self-documenting names:
`interval` -> `compact_interval`, `factor` -> `compact_bloat_factor`
(compact a slot at >= this x its live size), `shrink_to` -> `compact_target_ratio`
(shrink to this x live size), `min_bytes` -> `compact_min_bytes`. `trim_interval` kept.
- Drop the `active` keyword: auto-manage only ever runs at the depth-1 safepoint where
`n_active == 0`, so compacting "active" slots is a no-op concept here (it was a
`compact!`-only knob leaking into the auto API). `compact!`'s own kwargs are unchanged.
- The keyword defaults now come from flat `auto_manage_*` Preferences
(`auto_manage_compact_interval`, `auto_manage_trim_interval`,
`auto_manage_compact_bloat_factor`, `auto_manage_compact_target_ratio`,
`auto_manage_compact_min_bytes`), read once at load and coerced (`30` or `30.0` both
fine). So the common case is to set them once in `LocalPreferences.toml`, no startup
code; the keywords are the runtime override. `_AutoManageConfig` fields renamed to match.
Updates the legacy (<1.12) shim signature, all CPU/Metal/CUDA tests, the docstring, and
the Automatic Memory Management + Configuration docs. CPU auto suites green.
* test(auto_manage): cover the auto_manage_* Preference defaults
* docs(auto-manage): add a before/after schematic for compact vs trim
Add an ASCII before/after diagram in "What it does" making the two actions visual:
auto-compact shrinks the bloated buffer *within* one slot (a bar that gets shorter),
auto-trim drops *whole* slots beyond the recent peak (boxes that disappear). Renders in
Documenter's monospace code block — no Mermaid/plugin dependency. Also fixes the remaining
old kwarg names on this page (interval→compact_interval, factor→compact_bloat_factor,
min_bytes→compact_min_bytes) left over from the config rename.
* docs(auto-manage): define compact/trim first, then auto-* as timed+smart
Reframe "What it does": present `compact!` and `trim!` as the two primitives (with the
before/after schematic), then `auto_manage` as "run those automatically on a timer, a bit
smartly." The trim diagram now shows the ACTIVE slots (█ in use vs ░ idle-retained) so it's
clear trim keeps what's in use and drops the idle tail. The auto-* table adds the smart
part: auto-compact only touches bloated slots; auto-trim keeps the recent working-set PEAK
(not the instantaneous count) so a brief widening isn't thrashed.
mgyoo86
referenced
this pull request
Jul 1, 2026
Explicit release anchor for `@JuliaRegistrator register`. Project.toml is already at 0.3.8 (set in e7126ba); this dedicated empty commit is a clear target for the registration trigger comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
compact!, a manual operation that shrinks the over-allocated capacity of backing buffers in place — including buffers the caller is still using. It reclaims the peak capacity a pool retains after growing for a spike and then operating at a smaller steady size.Orthogonal to
trim!:trim!drops whole inactive slots;compact!shrinks the capacity of retained ones (active or inactive), keeping each slot ready for reuse at a right-sized buffer.API
force_gc=truerunsGC.gc(); on CUDA it additionally callsCUDA.reclaim().Semantics
A slot is bloated when its retained capacity is
≥ factor × used(its current logical size).compact!swaps such a backing's buffer for one of sizeceil(shrink_to × used)in place — keeping the backing object's identity, so held wrappers and views follow the swap (their:ref/:dataare re-synced and the old buffer is freed). A slot is compacted only if both gates pass: the ratio (factor) and the absolute reclaim ((capacity − target) × sizeof(T) ≥ min_bytes, default 1 MiB), so tiny buffers are not churned.active=true(default): also compact slots still in use —compact!'s reason for existing overtrim!. Thefactorgate means a normally-sized live array is never touched, only grossly over-allocated ones.active=false: opt out — inactive slots only.The live extent is tracked per slot (recorded in the shared claim path, covering both
acquire!andacquire_view!), so the summary is fully type-stable and the gate is allocation-free. Absurd kwargs (e.g.shrink_to=1e300) are safe no-ops, never throws — the shrink target is clamped to[used, capacity].Backends & versions
CPU + Metal + CUDA, on Julia 1.12+ (uses the
setfield!-based in-place buffer swap; on GPU the backing'sDataRefis swapped and wrappers re-synced via the existing refcount path).On Julia ≤1.11 (the legacy N-way-cache path)
compact!is a defined, exported no-op (zero summary + one-time@warn), so dependent packages can call it across the full[compat] juliarange without anUndefVarError.Tested
Inactive shrink, active-slot compaction with the held wrapper/view's data intact and writable through the swap, the
factor/min_bytesgates, GC reclaim of the old buffer (WeakRef), per-type / varargs /DisabledPoolforms, type stability (@inferred+return_types), self-heal after compaction, and a 24-context robustness sweep (empty pool, global scope,Bit/fallback types,@with_pool, repeated calls, zero-length, pathological kwargs).Metal verified on Apple Silicon (device buffer 1M→150 elements with the held array's data preserved); CUDA verified on device.
Files:
test/test_compact.jl(CPU) ·test/metal/test_compact.jl·test/cuda/test_compact.jl.Out of scope
Automatic / timer-based compaction at scope boundaries · usage-tracking & hysteresis · per-slot policy (outlier-only) · a real (non-no-op) legacy implementation.