(feat): auto-compact — timer-driven background capacity compaction by mgyoo86 · Pull Request #46 · ProjectTorreyPines/AdaptiveArrayPools.jl

mgyoo86 · 2026-06-30T04:20:02Z

Summary

Adds auto-compact: a GC-like background mechanism that periodically runs compact! to reclaim over-allocated backing buffers, without touching the hot path.
A single background Timer requests compaction; each pool services the request — the full compact! — at its next @with_pool boundary, where nothing is borrowed.
It turns the existing manual compact! into a fire-and-forget policy, the way the GC reclaims unused memory on its own cadence.

How it works

One global Timer (on a CPU thread) sweeps a WeakRef registry of live pools and sets each pool's @atomic _compact_requested flag.
The owner task consumes the flag at a scope entry (@with_pool / @with_pool :metal / @with_pool :cuda, at _current_depth == 1) and runs compact! there.

The Timer thread only ever writes one atomic flag — it never reads or mutates pool data, because pools are not thread-safe. All real work happens on the owner task at a safepoint, the same discipline the GC uses.
Servicing at scope entry (not exit) handles every exit type of the previous scope — normal / early return / break / continue / exception — with one hook point, and never runs compaction inside a finally during unwind.
A CPU pool and a GPU pool registered by the same task are flagged by one sweep and then compact independently, each at its own backend's scope boundary.

API & toggles

Two tiers, mirroring RUNTIME_CHECK:

# Tier 1 — compile-time master switch (Preference, default true)
const AUTO_COMPACT = @load_preference("auto_compact", true)   # false ⇒ feature fully DCE'd

# Tier 2 — runtime control of the background Timer
enable_auto_compact!(; interval = 30.0, factor = 10, shrink_to = 1.5,
                       min_bytes = 2^20, active = true)
disable_auto_compact!()
auto_compact_enabled() -> Bool

A slot is compacted iff its capacity is ≥ factor × last-use and the reclaim is ≥ min_bytes; it shrinks to ~shrink_to × last-use.
With AUTO_COMPACT on, __init__ auto-starts the Timer (default 30 s) — guarded against running during precompilation, so it never leaks a libuv handle into an extension's precompile worker.

Zero-overhead toggle

Default-on adds a single inlined @atomic :monotonic flag read per @with_pool entry (sub-nanosecond), gated by the compile-time AUTO_COMPACT.
Setting auto_compact = false in LocalPreferences.toml dead-code-eliminates the hook entirely — verified zero (@code_typed / @code_llvm show no _maybe_auto_compact!, @allocated == 0).

Backends & versions

CPU + Metal + CUDA, on Julia 1.12+.
On Julia ≤1.11 (the legacy N-way-cache path) the feature is a defined, exported no-op: AUTO_COMPACT == false, no-op register_auto_compact!/enable!/disable!, auto_compact_enabled() == false (one-time @warn) — so dependents can call the API across the full [compat] julia range without an UndefVarError.

Included perf: `compact!` copy-skip for inactive slots

compact! shrinks a bloated slot by allocating a right-sized backing and copying the live data.
For an inactive slot the old contents are dead (the next acquire! returns uninitialized memory the caller fills), so the copy is pure waste.
_compact_slot! now takes copy_live and copies only for active slots (slot ≤ n_active); inactive compaction is allocate-only — no memcpy on CPU, no GPU→GPU copyto! / kernel launch.
This is exactly the auto-compact case (every flagged slot is inactive). Applied to CPU, Metal, and CUDA; behavior-preserving for correct usage.

Tested

Flag field, registry sweep + dead-WeakRef pruning, _run_auto_compact!, the scope-entry hook (fires only at depth 1; preserved across nested scopes; no-op when clear; GPU ::Any fallback), the macro emitting the gated hook, timer lifecycle (enable/disable/replace, throwing-sweep survival), default-on integration (__init__ autostart, task-local auto-registration), deterministic end-to-end at a real @with_pool boundary including early-return/break and @safe_with_pool exception-unwind, the CPU↔GPU coexistence guarantee, and the per-slot copy-decision (active data preserved, inactive copy-skipped slot re-acquires correctly).
Files: test/test_auto_compact.jl (CPU) · test/test_auto_compact_legacy.jl (≤1.11 no-op) · test/metal/test_auto_compact.jl · test/cuda/test_auto_compact.jl · copy-skip cases in each test_compact.jl.

Out of scope

Idle-stats escalation (e.g. compact! → empty! for long-idle pools) · usage tracking / hysteresis · servicing a pool that never re-enters a scope (needs cooperative maintain!/release!) · Timer-thread direct compaction (unsafe without making the hot path lock-bearing).

`_compact_slot!` copied the `used` live elements into the new backing unconditionally. For INACTIVE slots (slot > n_active) that data is dead — the next `acquire!` returns uninitialized memory the caller fills — so the copy is pure waste. Gate it on a per-slot active flag: active slots (1:n_active) still copy to preserve held data; inactive slots allocate the right-sized buffer with no copy. Eliminates compact!'s only non-sub-µs case: reclaiming a large-last-extent inactive slot drops from ~138µs to ~166ns (alloc-only). Active-slot data preservation is unchanged — guarded by existing tests plus a new per-slot characterization test (active slot 1 data preserved, inactive slot 2 shrinks + re-acquires correctly under active=true). Julia's `Memory` cannot be shrunk in place, so alloc+free stays the only real reclaim path; this just drops the needless memcpy. CPU only; the Metal/CUDA `_compact_slot!` methods carry the same opportunity (follow-up).

Foundation for timer-driven background compaction — the data layer only (no Timer, no macro hook yet; phases 2-3 follow). Every piece is exercised directly by test_auto_compact.jl. - AdaptiveArrayPool gains an `@atomic _compact_requested::Bool` field: the cross-thread handoff cell (Timer writes, owner reads+resets at its safepoint). 1 byte; both positional constructors updated; reset on pool migration. - New src/auto_compact.jl: * AUTO_COMPACT — compile-time Preference (Tier 1), default off → zero cost. * _AUTO_COMPACT_REGISTRY (Vector{WeakRef}) + _AUTO_COMPACT_LOCK. * _AutoCompactConfig + defaults mirroring compact! (factor=10, shrink_to=1.5, min_bytes=2^20, active=true). * register_auto_compact!, _auto_compact_sweep! (flag live pools, prune dead), _run_auto_compact! (reset-then-compact; depth guard is the caller's). CPU/modern path only.

Runtime control (Tier 2) for the background compaction timer: - enable_auto_compact!(; interval=30, factor, shrink_to, min_bytes, active): set config + (re)start the single global Timer. Warns when the `auto_compact` Preference is off (hot-path compiled out) but still starts the timer, so manual registration / testing works. - disable_auto_compact!() — stop the timer (idempotent). - auto_compact_enabled() -> Bool. - _safe_auto_compact_sweep!: try/catch around the sweep so a throwing callback cannot silently kill the Timer. - __init__ auto-starts with defaults when AUTO_COMPACT is on (dead-code-eliminated when off → no timer on a normal load). No macro/hot-path integration yet (phase 3). Tests cover start→flag→stop lifecycle, config-setting, timer replacement, disable idempotency, and the error-swallowing wrapper.

@inline

Wire the timer's flag into the hot path, fully gated by AUTO_COMPACT: - _maybe_auto_compact!(pool): @inline scope-exit hook body — at the _current_depth == 1 safepoint, if the flag is set, run the compaction (reset-then-compact). An ::Any fallback no-ops for non-CPU pools, keeping the SHARED @with_pool codegen safe (GPU auto-compact is a future phase). - macros.jl: every @with_pool exit (block + function, safe + non-safe) emits `AUTO_COMPACT && _maybe_auto_compact!(pool)` after the normal rewind. Injected once per generator by wrapping rewind_call, so early return/break sites are untouched (best-effort: catches the next normal exit at depth 1). - get_task_local_pool registers new task-local pools (AUTO_COMPACT-gated). Zero overhead when off: AUTO_COMPACT is a const false, so the hook is constant-folded out of the typed IR and LLVM entirely (verified: no trace in either, and @allocated == 0 across a @with_pool loop). ~0.6 ns inlined check when on. CPU/modern path only.

Make auto-compaction the default and complete the cross-version API. - AUTO_COMPACT Preference now defaults to `true` (opt-out): out of the box the scope-exit hook is compiled in (~0.6 ns/exit), task-local pools auto-register, and __init__ auto-starts the 30s timer. Set `auto_compact = false` to compile it out entirely (full DCE → zero hot-path cost). Removed a stray debug println. - src/legacy/auto_compact.jl (new): `const AUTO_COMPACT = false` + register_/_maybe_ no-ops + enable!/disable!/enabled no-op shims, so the SHARED macros.jl / task_local_pool.jl references resolve and the @with_pool hook DCEs on < 1.12. Fixes the `UndefVarError: AUTO_COMPACT` seen on the legacy path. - Export enable_auto_compact!, disable_auto_compact!, auto_compact_enabled, AUTO_COMPACT. - runtests.jl disables the auto-started timer up front for deterministic tests. - test_auto_compact.jl reworked for default-on: drops obsolete warn-asserts; adds __init__-autostart, auto-registration, and a deterministic end-to-end test that the macro hook compacts at a @with_pool boundary. New test_auto_compact_legacy.jl. CPU/modern path runs the real hook; < 1.12 is a portable no-op.

__init__ auto-started the background Timer whenever AUTO_COMPACT is on. When a CUDA/Metal extension is precompiled, its worker loads AdaptiveArrayPools and runs __init__ → the live Timer leaves an open libuv handle → the precompile worker hangs with "unfinished IO handles with timer events" (surfaced by the full suite, which precompiles both extensions). Guard the auto-start with `ccall(:jl_generating_output, Cint, ()) == 0` so no Timer is started during precompile/sysimage output, and atexit-close it for tidy shutdown. Normal processes still get default-on — verified: Metal + CUDA extensions now precompile cleanly and auto_compact is enabled=true at runtime. Also document in enable_auto_compact! that auto-compaction targets the task-local pool only and runs at @with_pool exits.

Adopt a scope-ENTRY hook (check the flag before the @with_pool checkpoint, at depth==1) instead of the scope-exit hook. One hook point services a pending request when the NEXT @with_pool is entered, regardless of how the previous scope exited — normal, early return, break, continue, or exception — and never runs compaction inside a `finally` during unwind. This resolves the adversarial review's two must-fix findings (early-exit paths skipped the hook; finally-path compaction could mask a user exception) by construction. Additional review hardening: - Owner-side compaction wrapped in try/catch: a background-maintenance failure is logged and the cycle skipped; it never surfaces at the @with_pool boundary. - Atomic flag uses :monotonic — a one-way eventual-visibility signal with no cross-field ordering invariant; avoids seq-cst barriers on weak-ordered CPUs (e.g. Apple Silicon), the exact per-entry cost the hook minimizes. - Config snapshot + timer lifecycle guarded by the registry lock (no torn config read across threads; no leaked/double-closed Timer from concurrent enable/disable). - enable_auto_compact! warn gains maxlog=1; register_auto_compact! narrowed to ::AdaptiveArrayPool so a mis-registration fails at the call site. - Document the zero-alloc trade-off: a @with_pool that services a request allocates the right-sized backing, so it is not strictly zero-alloc; set auto_compact=false for a guaranteed-zero-alloc hot path. New tests: early return / break service the request at the next entry; @safe_with_pool fires the hook during exception unwind and preserves the exception.

Mirror the CPU auto-compact wiring onto the Metal backend so a `MetalAdaptiveArrayPool` is flagged by the single background Timer and runs the full `compact!` at its next `@with_pool :metal` entry, exactly as CPU pools do. Base module (shared across backends): - Broaden `register_auto_compact!`, the `_auto_compact_sweep!` flag write, and `_run_auto_compact!` from `AdaptiveArrayPool` to the `AbstractArrayPool` supertype. One registry, one sweep, one runner now serve every backend; `compact!` takes the same kwargs everywhere, so the runner is unchanged. Metal extension: - Add the `@atomic _compact_requested::Bool` field to `MetalAdaptiveArrayPool` (+ constructor) — the cross-thread handoff flag, parity with CPU. - New `_maybe_auto_compact!(::MetalAdaptiveArrayPool)`: services the flag only at `_current_depth == 1`, reusing the base `_run_auto_compact!`. The scope-entry hook is already emitted by the shared `@with_pool` codegen; before this it fell to the `::Any` no-op. - Register each task-local Metal pool (per device) for auto-compaction in the slow path, gated by the compile-time `AUTO_COMPACT` (DCE'd when off). Coexistence: a CPU pool and a Metal pool share the one CPU-side Timer. A single sweep flags both; each then compacts independently at its own scope boundary, and servicing one never touches the other's flag or buffers — pinned by a dedicated test. Full Metal suite + new `test/metal/test_auto_compact.jl` (30 tests) green on Apple Silicon.

Mirror the Metal auto-compact wiring onto the CUDA backend (the base-module support — one registry, one Timer, `AbstractArrayPool`-typed sweep/runner — was already broadened in the Metal commit, so this is purely the CUDA-side three edits plus tests). - Add the `@atomic _compact_requested::Bool` field to `CuAdaptiveArrayPool` (+ both construction paths: the inner ctor and the `_transfer_cuda_pool` migration ctor) — the cross-thread handoff flag, parity with CPU/Metal. - New `_maybe_auto_compact!(::CuAdaptiveArrayPool)`: services the flag only at `_current_depth == 1`, reusing the base `_run_auto_compact!`. The scope-entry hook is already emitted by the shared `@with_pool :cuda` codegen; before this it fell to the `::Any` no-op. - Register each task-local CUDA pool (per device) for auto-compaction in the slow path, gated by the compile-time `AUTO_COMPACT` (DCE'd when off). Tests: test/cuda/test_auto_compact.jl mirrors the Metal suite (field, registry sweep, scope-entry hook, end-to-end @with_pool :cuda, and a CPU+CUDA coexistence test pinning that one sweep flags both and each compacts independently at its own boundary). Validated locally as far as a non-NVIDIA host allows: the CUDA extension precompiles cleanly and reflection confirms the field placement, the concrete _maybe_auto_compact! method, and AbstractArrayPool dispatch. The GPU runtime assertions are validated on the remote CUDA box.

The Metal and CUDA `test_auto_compact.jl` files defined `_clear_registry!` / `_registry_len` (and `_cpucap`) as top-level functions in `Main`, colliding with the identically-named top-level helpers in the CPU `test_auto_compact.jl` when the full suite loads several backends' files together: WARNING: Method definition _clear_registry!() in module Main at test/test_auto_compact.jl:29 overwritten at test/cuda/test_auto_compact.jl:22. Move each GPU file's helpers INSIDE its `@testset ... begin` (a local scope, the same convention the `test_compact.jl` files already use), so they no longer define `Main` methods. The CPU file stays the sole top-level definer — no collision. Warning only; all tests passed before and after. Verified: 0 "overwritten" warnings when the CPU + Metal files load into one Main, both suites green.

…ot copy) CPU `_compact_slot!` takes a `copy_live::Bool` and copies the live `used` elements only for ACTIVE slots (`slot <= n_active`); an inactive slot's contents are dead (the next `acquire!` returns uninitialized memory the caller fills), so the copy is skipped — inactive compaction is allocate-only. The GPU backends only had the weaker `used > 0` skip (catches empty slots only), so an inactive slot that was used still paid a wasted GPU→GPU `copyto!` of `used` elements — exactly the auto-compact case, where every flagged slot is inactive. Bring Metal and CUDA `_compact_slot!` to full CPU parity: add the `copy_live` parameter (`copy_live && copyto!`), and have `_maybe_compact_slot!` pass `slot <= tp.n_active`. Behavior-preserving for correct usage (inactive contents are dead, never read) — it removes a device copy / kernel launch per inactive compacted slot. Tests: add the per-slot copy-decision testset to test/metal/test_compact.jl and test/cuda/test_compact.jl (mirror of the CPU one) — two bloated slots, hold only slot 1, `compact!(active=true)`: both shrink, the held slot's data survives, and the inactive (copy-skipped) slot still re-acquires correctly. Full Metal suite green on Apple Silicon; CUDA validated on the remote box.

codecov · 2026-06-30T04:23:12Z

Codecov Report

❌ Patch coverage is 96.20253% with 3 lines in your changes missing coverage. Please review.
✅ Project coverage is 95.74%. Comparing base (fff5d38) to head (988c50f).

Files with missing lines	Patch %	Lines
src/auto_compact.jl	95.38%	3 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master      #46   +/-   ##
=======================================
  Coverage   95.73%   95.74%           
=======================================
  Files          14       16    +2     
  Lines        3493     3569   +76     
=======================================
+ Hits         3344     3417   +73     
- Misses        149      152    +3

Files with missing lines	Coverage Δ
src/AdaptiveArrayPools.jl	`100.00% <ø> (ø)`
src/legacy/auto_compact.jl	`100.00% <100.00%> (ø)`
src/macros.jl	`96.79% <100.00%> (+<0.01%)`	⬆️
src/state.jl	`96.52% <100.00%> (+<0.01%)`	⬆️
src/task_local_pool.jl	`100.00% <100.00%> (ø)`
src/types.jl	`85.52% <ø> (ø)`
src/auto_compact.jl	`95.38% <95.38%> (ø)`

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@warn

Two non-failing warnings in the full suite looked like errors: 1. `WARNING: Method definition _cap(Any) ... overwritten` — test_auto_compact.jl defined a top-level `_cap` helper in `Main`, colliding with the identically-named top-level helper in test_compact.jl. Same class as 0d9682a (GPU files), but CPU↔CPU, which that commit missed (test_compact.jl is an independent definer of `_cap`, not just test_auto_compact.jl). Scope the CPU helpers (`_cap`, `_clear_registry!`, `_registry_len`, `_inactive_bloated`) inside the testset. 2. The "auto-compact sweep failed (timer continues)" @warn is emitted by the DELIBERATE negative test that pushes a non-pool into the registry to prove the sweep wrapper catches the throw (so the Timer survives). The test passes; the warning just read as a failure. Wrap that one call in a NullLogger so the expected warning is silenced — the no-rethrow assertion is unchanged. Verified: 0 "overwritten" and 0 "sweep failed" lines when test_compact.jl + test_auto_compact.jl load together; compact! 105/105, auto_compact 54/54.

Copilot

Pull request overview

This PR adds timer-driven background auto-compaction to reclaim over-allocated pool backing buffers on a periodic cadence, while keeping the hot path limited to a single gated atomic flag check at @with_pool scope entry. It also improves compact! by skipping unnecessary copies for inactive slots across CPU/Metal/CUDA.

Changes:

Introduces AUTO_COMPACT (Preferences-gated) plus enable_auto_compact! / disable_auto_compact! / auto_compact_enabled() and a global Timer + WeakRef registry to request compaction.
Wires a scope-entry hook into @with_pool (CPU/GPU) and adds per-pool atomic _compact_requested fields + task-local auto-registration.
Optimizes compaction by copying live data only for active slots; adds extensive CPU/Metal/CUDA + legacy-noop tests and integrates them into test runners.

Reviewed changes

Copilot reviewed 27 out of 27 changed files in this pull request and generated 10 comments.

Show a summary per file

File	Description
test/test_compact.jl	Adds coverage for per-slot active/inactive copy behavior during `compact!(; active=true)`.
test/test_auto_compact.jl	New CPU auto-compact test suite covering registry sweep, hook behavior, timer lifecycle, and end-to-end behavior.
test/test_auto_compact_legacy.jl	New legacy-path tests asserting the exported auto-compact API is a defined no-op on Julia < 1.12.
test/runtests.jl	Disables auto-compact timer for deterministic tests; includes new auto-compact test files per Julia version.
test/metal/test_compact.jl	Adds Metal coverage for active/inactive per-slot copy decision.
test/metal/test_auto_compact.jl	New Metal auto-compact parity tests, including CPU+Metal coexistence behavior.
test/metal/runtests.jl	Includes Metal auto-compact tests in the Metal test runner.
test/cuda/test_compact.jl	Adds CUDA coverage for active/inactive per-slot copy decision.
test/cuda/test_auto_compact.jl	New CUDA auto-compact parity tests, including CPU+CUDA coexistence behavior.
test/cuda/runtests.jl	Includes CUDA auto-compact tests in the CUDA test runner.
src/types.jl	Adds atomic `_compact_requested` flag to `AdaptiveArrayPool` and initializes/resets it in constructors/migration.
src/task_local_pool.jl	Auto-registers new task-local CPU pools for auto-compaction when `AUTO_COMPACT` is enabled.
src/state.jl	Extends `_compact_slot!` with `copy_live` and gates copying to active slots only.
src/macros.jl	Emits a Preferences-gated scope-entry auto-compact hook in `@with_pool` expansions.
src/legacy/auto_compact.jl	Adds legacy no-op shims for the auto-compact API and hook references on Julia < 1.12.
src/auto_compact.jl	Implements registry + timer sweep + scope-entry servicing hook + runtime enable/disable API.
src/AdaptiveArrayPools.jl	Exports auto-compact API, includes modern vs legacy auto-compact implementations by Julia version.
ext/AdaptiveArrayPoolsMetalExt/types.jl	Adds atomic `_compact_requested` flag to Metal pool type and initializes it.
ext/AdaptiveArrayPoolsMetalExt/task_local_pool.jl	Auto-registers new (task, device) Metal pools when `AUTO_COMPACT` is enabled.
ext/AdaptiveArrayPoolsMetalExt/state.jl	Adds `copy_live` support to Metal `_compact_slot!` and gates copying to active slots.
ext/AdaptiveArrayPoolsMetalExt/auto_compact.jl	Adds Metal-specific `_maybe_auto_compact!` method to service flagged requests at Metal scope entry.
ext/AdaptiveArrayPoolsMetalExt/AdaptiveArrayPoolsMetalExt.jl	Includes Metal auto-compact implementation file in the extension module.
ext/AdaptiveArrayPoolsCUDAExt/types.jl	Adds atomic `_compact_requested` flag to CUDA pool type and initializes/resets it in constructors/migration.
ext/AdaptiveArrayPoolsCUDAExt/task_local_pool.jl	Auto-registers new (task, device) CUDA pools when `AUTO_COMPACT` is enabled.
ext/AdaptiveArrayPoolsCUDAExt/state.jl	Adds `copy_live` support to CUDA `_compact_slot!` and gates copying to active slots.
ext/AdaptiveArrayPoolsCUDAExt/auto_compact.jl	Adds CUDA-specific `_maybe_auto_compact!` method to service flagged requests at CUDA scope entry.
ext/AdaptiveArrayPoolsCUDAExt/AdaptiveArrayPoolsCUDAExt.jl	Includes CUDA auto-compact implementation file in the extension module.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot review on PR #46 (all valid): - register_auto_compact!(::AbstractArrayPool) could accept a flagless pool (DisabledPool, or a foreign AbstractArrayPool subtype) — registering one makes the sweep throw on its missing `_compact_requested` and abort the cycle, starving later pools. Guard with `hasfield(typeof(pool), :_compact_requested)` (const-folds per concrete type); add a regression test that a DisabledPool is silently skipped and a real pool registered alongside it still gets flagged. - Docstrings/warning for the `AUTO_COMPACT=false` case were misleading: with the Preference off the `@with_pool` hook is compiled out, so the timer's flags are never serviced — manual `register_auto_compact!` does NOT make it effective. Reword the `enable_auto_compact!` docstring + warning to say so, and soften "hand-made pool is never registered" to "not auto-registered (register explicitly)". - legacy/auto_compact.jl + test_auto_compact_legacy.jl said "scope-exit hook" — stale from the exit-hook era; the hook is at scope ENTRY now. Also fix a test-env load error (surfaced by the CUDA run): the warning-suppression helper used `using Logging`, which isn't a declared test dep. Source `with_logger` / `NullLogger` from `Base.CoreLogging` instead — built into Base, no dependency needed. CPU auto_compact suite green (55 tests).

Codex review (PR #46, P1): with RUNTIME_CHECK on (S/R=1), releasing a slot poisons its backing (NaN/typemax) AND shrinks the logical length to 0 (CPU `resize!(v,0)`; GPU `_resize_to_fit!(v,0)`, device memory retained), so an escaped `acquire_view!` view reads poison instead of live data. But `_maybe_compact_slot!` still saw such a slot as "bloated" (large retained capacity, nonzero recorded extent) and — with the inactive copy-skip (`copy_live=false`) — swapped in a fresh *uninitialized* buffer, UNDOING the poison: the escaped view would then read plausible memory instead of NaN. `compact!(; active=false)` (and auto-compact at depth 1, which sees every slot inactive) silently defeated escape detection. Fix: skip compacting a poisoned slot. Poison and the length-0 shrink are set together in `_invalidate_released_slots!`, and an unchecked slot keeps its logical length, so `length(v) == 0 && used > 0` is an exact marker for "poisoned". Add `length(v) == 0 && return 0` to `_maybe_compact_slot!` on CPU, Metal, and CUDA — no signature threading, and S=0 (the zero-overhead default) is unaffected (released slots there keep their length, so the guard never fires). The slot compacts normally once re-acquired. Active held slots (copy_live=true) are untouched. Tests: RUNTIME_CHECK=1 regression in test/test_compact.jl (proved the bug — current code shrank a poisoned 1M slot to 150) + test/metal/test_compact.jl + test/cuda/test_compact.jl. Full Metal suite green on Apple Silicon (incl. the R=1 case); CPU green; CUDA validated on the remote box.

@warn

* refactor: rename auto_compact → auto_manage (unify the maintenance engine) The background-maintenance engine is about to gain a second action (auto-trim), so "compact" undersells it. Rename the unreleased umbrella `auto_compact` → `auto_manage` across CPU, legacy, Metal, CUDA, and tests — const `AUTO_COMPACT`→`AUTO_MANAGE`, Preference `auto_compact`→`auto_manage`, `enable_/disable_/…_enabled`, the engine internals (`_run_/_maybe_/register_/_…_sweep!`, `_AUTO_MANAGE_{REGISTRY,TIMER,CONFIG,LOCK}`, `_AutoManageConfig`), and `src/auto_compact.jl`→`src/auto_manage.jl` (+ legacy + exts + test files). Pure rename — no behavior change. The compact *action* is untouched: `compact!`, the `_compact_requested` flag, and "compaction" wording all stay. Safe to rename: no released/registered version ships `auto_compact` (latest tag v0.3.7 predates #44/#45/#46; version still 0.3.7). Validated: renamed CPU + legacy tests green; full Metal suite green (ext rename loads); runic clean; 0 leftover `auto_compact`/`AUTO_COMPACT`/`AutoCompact` references. CUDA rename validated on the remote box. * feat(auto_manage): auto-trim — periodic working-set tail reclamation (CPU) Second action of the auto_manage engine: every `trim_interval`, trim each type's slot tail down to its recent working-set peak, reclaiming the inactive tail that `compact!` only shrinks. Catches the common case of a pool that peaked wide then runs narrow (still entered, so whole-pool-idle schemes never reclaim it). - `_ac_peak_n_active::Int` on TypedPool + BitTypedPool: peak `n_active` since the last trim. One branchless `max` in the claim paths (`_claim_slot!`, `get_bitarray!`), gated by the compile-time `AUTO_MANAGE` const → DCE'd to zero when off (measured ~0.15 ns/acquire, zero-alloc when on). Reset to 0 at each auto-trim, so it doubles as the period accumulator (trim targets the *recent* peak, never the all-time max). - `@atomic _trim_requested::Bool` on AdaptiveArrayPool (+ both ctor paths), parallel to `_compact_requested`. The Timer sets it every `trim_every_ticks = round(trim_interval/interval)` sweeps via a global tick counter. - `_trim_to!(tp, keep)`: generalizes `trim!` (truncate to `keep`, clamped ≥ n_active). - `_run_auto_manage!`: services `_trim_requested` (→ `_auto_trim!`, trim each type to its peak then reset) then `_compact_requested` (→ compact!), each guarded by its flag. The entry hook is unchanged (still triggers on `_compact_requested`, set on the same sweep). `hasfield` guards keep it backend-safe before GPU parity lands. - `enable_auto_manage!(; trim_interval = 120.0)`; `Inf` disables auto-trim. Tests (test/test_auto_trim.jl): peak tracking, `_trim_to!`, auto-trim to the recent peak / to 0 for an unused type, no-op when unflagged, timer cadence, `Inf` disable, zero-alloc hot path, and a RUNTIME_CHECK=1 regression — auto-trim drops the slot reference without swapping the buffer, so an escaped view's poison survives (unlike the compact! poison-undo bug fixed in 988c50f). Tier-1 green. * feat(auto_manage): auto-trim Metal parity Mirror the CPU auto-trim data layer (commit 24586d9) onto the Metal backend. The service layer (`_trim_to!`, `_auto_trim!`, `_run_auto_manage!`) is generic over the abstract pool supertypes and already worked for Metal, so this is purely the two telemetry fields plus the claim-path `max`: - `_ac_peak_n_active::Int` on MetalTypedPool: peak `n_active` since the last trim. One branchless `max` in both `_metal_claim_slot!` methods, gated by the compile-time AUTO_MANAGE const -> DCE'd to nothing when off (one Int write when on). - `@atomic _trim_requested::Bool` on MetalAdaptiveArrayPool (+ ctor), parallel to `_compact_requested`. The base-module Timer sweep's `hasfield` guard now const-folds true for Metal pools, so the sweep flags trim every `trim_interval`; the unchanged entry hook (fires on `_compact_requested`, set on the same sweep) lets the generic `_run_auto_manage!` service both at the `@with_pool :metal` safepoint. Tests (test/metal/test_auto_trim.jl): the CPU suite mirrored for Metal -- peak tracking through `_metal_claim_slot!`, `_trim_to!` truncation + regrow, auto-trim to the recent peak / to 0 for an unused type, no-op when unflagged, an R=1 no-buffer-swap guard (the dropped slot's MtlArray DataRef is left intact, unlike compact!), and the sweep flagging `_trim_requested` on a registered Metal pool. Full Metal suite green locally (57s). * feat(auto_manage): auto-trim CUDA parity Mirror the CPU/Metal auto-trim data layer onto the CUDA backend (commits 24586d9, ab9be38). The service layer (`_trim_to!`, `_auto_trim!`, `_run_auto_manage!`) is generic over the abstract pool supertypes and already worked for CUDA, so this is purely the two telemetry fields plus the claim-path `max`: - `_ac_peak_n_active::Int` on CuTypedPool: peak `n_active` since the last trim. One branchless `max` in both `_cuda_claim_slot!` methods, gated by the compile-time AUTO_MANAGE const -> DCE'd to nothing when off. The operands are host-side `Int` fields (NOT device arrays), so this is a CPU integer op, not a GPU scalar op. - `@atomic _trim_requested::Bool` on CuAdaptiveArrayPool (+ both the main ctor and the `_transfer_cuda_pool` migration ctor, which resets it like `_compact_requested`), parallel to `_compact_requested`. The base-module Timer sweep's `hasfield` guard now const-folds true for CUDA pools, so the sweep flags trim every `trim_interval`; the unchanged entry hook (fires on `_compact_requested`, set on the same sweep) lets the generic `_run_auto_manage!` service both at the `@with_pool :cuda` safepoint. Tests (test/cuda/test_auto_trim.jl): the CPU/Metal suite mirrored for CUDA -- peak tracking through `_cuda_claim_slot!`, `_trim_to!` truncation + regrow, auto-trim to the recent peak / to 0 for an unused type, no-op when unflagged, an S=1 no-buffer-swap guard (the dropped slot's CuArray DataRef is left intact, unlike compact!), and the sweep flagging `_trim_requested` on a registered CUDA pool. Validated locally without a GPU via extension precompile + struct/ctor-arity reflection (fieldcount 7 / 18); the device-level suite runs on the CUDA box. * test(auto_manage): end-to-end integrity + real-time cadence Close the gap between the unit suites (which test the mechanism in isolation -- set a flag, call the service; the depth-1 safepoint no-op) and the safety guarantee that actually matters: the real background Timer firing repeatedly mid-computation must never corrupt a result. test/test_auto_integration.jl: - A. Differential integrity: a deterministic, pool-heavy simulation (varying array size -> compact bloat; varying concurrency width -> trim tail; nested depth-2 scope) run with auto-manage OFF then ON (real Timer at 0.01s, ~40 sweeps over the run). Asserts the two result vectors are BIT-IDENTICAL -- auto-compact/auto-trim reclaim memory between scopes, never change a computed value. Plus self-heal-after-reclaim correctness, and measurable reclamation (slot tail + byte bloat both shrink). - B. Real-time cadence: over real elapsed seconds, asserts the Timer flags compaction every tick and auto-trim only every K-th tick (K = round(trim_interval/interval)), observed without servicing so the raw cadence is visible. Timing margins match/exceed the existing sleep-based timer tests; verified stable across repeated local runs (9/9 testset, ~5s). CPU only for now; Metal/CUDA mirrors to follow. * test(auto_manage): GPU integrity + real-time cadence (Metal + CUDA) Mirror the CPU end-to-end integration suite (006552c) onto Metal and CUDA. Same two guarantees -- (A) the real Timer firing repeatedly mid-computation produces BIT-IDENTICAL results vs auto-manage off, and (B) the compact-every-tick / trim-every-K-th cadence over real elapsed seconds -- adapted for GPU: - No element-wise writes (that is GPU scalar indexing): each slot is fill!ed with a small integer-valued Float32 kept under 2^24 * value, so the on-device `sum` is an exact integer independent of the parallel reduction order (bit-reproducible run-to-run, verified), while fill! over the whole buffer keeps the test sensitive to any aliasing/corruption. Metal: full suite green locally (integration testset 9/9, 4.1s). CUDA: identical mirror, parse-checked locally; the device suite runs on the GPU box. * refactor: rename _ac_peak_n_active → _am_peak_n_active (auto_manage namespace) The `_ac_` prefix was a leftover from the pre-rename `auto_compact` era; the field is auto-manage telemetry (consumed by auto-trim), so `_am_` matches AUTO_MANAGE and the `auto_manage` umbrella naming used everywhere else. Pure mechanical rename of the private field across CPU + Metal + CUDA + tests — no public API, branch unreleased. CPU auto suites green; Metal full suite validating. * fix(compact): active compact! no longer strands reshape! aliases reshape!(pool, A, dims) with ndims(A) != length(dims) stores its reshaped wrapper in a SEPARATE placeholder slot while aliasing A's backing memory. _compact_slot! re-pointed only the wrappers cached AT the compacted slot index, so the cross-slot reshape alias was left on the freed backing after an active=true compact! -- A followed the new memory while the reshape view silently diverged (reproduced with default compact! settings on a 1.6 MB slot, not just contrived parameters). Fix (all three backends): capture the slot's backing identity BEFORE the swap and re-point EVERY cached wrapper that shares it, across all slots -- not just wrappers[slot]: - CPU: match by Memory identity (getfield(w,:ref).mem === old_mem), setfield!(:ref). - Metal/CUDA: match by DataRef refcount identity (getfield(w,:data).rc === old_rc) and migrate via the refcount-balanced _update_*_wrapper_data! helper -- the same .data.rc identity reshape! itself uses to decide buffer sharing. The scan runs in the cold compaction path (which already allocates a backing + copyto!s), so the extra work is negligible; every pool wrapper points at offset 0 of its backing, so Memory/DataRef identity is a precise, offset-safe alias key. TDD: RED test pinned per backend (test/{,metal/,cuda/}test_compact.jl) -- write through one view, read the other, plus a shared-backing assertion -- failed before, passes after. CPU full suite + Metal full suite green; CUDA precompiled + parse-checked (device suite remote). * docs: document auto_manage (Automatic Memory Management page + API/config) Add a Features page covering the background auto-compact/auto-trim engine — what it reclaims, the @with_pool safepoint model (why it can't change results), default-on, the enable/disable knobs, the zero-alloc trade-off + auto_manage=false escape hatch, and manual compact!/trim!. Add compact! + the auto_manage symbols to the API reference tables and an auto_manage preference section to Configuration. Also fix a pre-existing broken [Safety](safety.md) link in configuration.md (→ ../safety/overview.md). * fix(auto_manage): address PR #47 review + raise patch coverage to 100% Review fixes (GitHub Copilot): - Legacy (Julia <1.12) `enable_auto_manage!` shim now accepts (and ignores) the new `trim_interval` keyword, so the public API is call-compatible across all supported Julia (previously a MethodError on old Julia when passing trim_interval). - Refresh the `enable_auto_manage!` docstring: show `trim_interval`, document auto-trim, generalize the "what gets managed" note (compaction + trim), drop the typo. - Fix rename artifacts from the auto_compact->auto_manage token replace: "auto-manageion" -> "auto-management" / "auto-manageed" -> "auto-managed" (15 comments across 10 files), and the misleading "Auto-Compact" file/section headers -> "Auto-Manage". Patch coverage (was 94.07% on src/auto_manage.jl, the only file below 100%): - Cover the `_auto_trim!` `others`-type loop with a non-fixed-slot (UInt8) auto-trim test. - Cover the two `_run_auto_manage!` try/catch+@warn defensive branches by parking a non-pool object in both others collections (values(others) for trim, _others_values for CPU compact) — mirrors the existing throwing-sweep test. src/auto_manage.jl now 100% (90/90). * (release): bump version to v0.3.8 * feat(auto_manage): descriptive config kwargs + LocalPreferences defaults Polish the public tuning API before it ships in v0.3.8: - Rename `enable_auto_manage!` keywords to self-documenting names: `interval` -> `compact_interval`, `factor` -> `compact_bloat_factor` (compact a slot at >= this x its live size), `shrink_to` -> `compact_target_ratio` (shrink to this x live size), `min_bytes` -> `compact_min_bytes`. `trim_interval` kept. - Drop the `active` keyword: auto-manage only ever runs at the depth-1 safepoint where `n_active == 0`, so compacting "active" slots is a no-op concept here (it was a `compact!`-only knob leaking into the auto API). `compact!`'s own kwargs are unchanged. - The keyword defaults now come from flat `auto_manage_*` Preferences (`auto_manage_compact_interval`, `auto_manage_trim_interval`, `auto_manage_compact_bloat_factor`, `auto_manage_compact_target_ratio`, `auto_manage_compact_min_bytes`), read once at load and coerced (`30` or `30.0` both fine). So the common case is to set them once in `LocalPreferences.toml`, no startup code; the keywords are the runtime override. `_AutoManageConfig` fields renamed to match. Updates the legacy (<1.12) shim signature, all CPU/Metal/CUDA tests, the docstring, and the Automatic Memory Management + Configuration docs. CPU auto suites green. * test(auto_manage): cover the auto_manage_* Preference defaults * docs(auto-manage): add a before/after schematic for compact vs trim Add an ASCII before/after diagram in "What it does" making the two actions visual: auto-compact shrinks the bloated buffer *within* one slot (a bar that gets shorter), auto-trim drops *whole* slots beyond the recent peak (boxes that disappear). Renders in Documenter's monospace code block — no Mermaid/plugin dependency. Also fixes the remaining old kwarg names on this page (interval→compact_interval, factor→compact_bloat_factor, min_bytes→compact_min_bytes) left over from the config rename. * docs(auto-manage): define compact/trim first, then auto-* as timed+smart Reframe "What it does": present `compact!` and `trim!` as the two primitives (with the before/after schematic), then `auto_manage` as "run those automatically on a timer, a bit smartly." The trim diagram now shows the ACTIVE slots (█ in use vs ░ idle-retained) so it's clear trim keeps what's in use and drops the idle tail. The auto-* table adds the smart part: auto-compact only touches bloated slots; auto-trim keeps the recent working-set PEAK (not the instantaneous count) so a brief widening isn't thrashed.

Explicit release anchor for `@JuliaRegistrator register`. Project.toml is already at 0.3.8 (set in e7126ba); this dedicated empty commit is a clear target for the registration trigger comment.

mgyoo86 added 11 commits June 29, 2026 14:13

mgyoo86 requested a review from Copilot June 30, 2026 04:20

Copilot started reviewing on behalf of mgyoo86 June 30, 2026 04:20

View session

Copilot AI reviewed Jun 30, 2026

View reviewed changes

mgyoo86 added 2 commits June 29, 2026 21:34

mgyoo86 merged commit 5abcda6 into master Jun 30, 2026
14 checks passed

mgyoo86 deleted the feat/auto_compact branch June 30, 2026 05:45

mgyoo86 referenced this pull request Jul 1, 2026

version bump to v0.3.8

f8fa236

Explicit release anchor for `@JuliaRegistrator register`. Project.toml is already at 0.3.8 (set in e7126ba); this dedicated empty commit is a clear target for the registration trigger comment.

Navigation Menu

Repository navigation

(feat): auto-compact — timer-driven background capacity compaction - #46

(feat): auto-compact — timer-driven background capacity compaction#46
mgyoo86 merged 14 commits into
masterfrom
feat/auto_compact

mgyoo86 commented Jun 30, 2026

Uh oh!

codecov Bot commented Jun 30, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

mgyoo86 commented Jun 30, 2026

Summary

How it works

API & toggles

Zero-overhead toggle

Backends & versions

Included perf: compact! copy-skip for inactive slots

Tested

Out of scope

Uh oh!

codecov Bot commented Jun 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Included perf: `compact!` copy-skip for inactive slots

codecov Bot commented Jun 30, 2026 •

edited

Loading