Repository navigation
(feat): auto-trim — automatically reclaim idle pool memory - #47
Conversation
…gine)
The background-maintenance engine is about to gain a second action (auto-trim), so
"compact" undersells it. Rename the unreleased umbrella `auto_compact` → `auto_manage`
across CPU, legacy, Metal, CUDA, and tests — const `AUTO_COMPACT`→`AUTO_MANAGE`,
Preference `auto_compact`→`auto_manage`, `enable_/disable_/…_enabled`, the engine
internals (`_run_/_maybe_/register_/_…_sweep!`, `_AUTO_MANAGE_{REGISTRY,TIMER,CONFIG,LOCK}`,
`_AutoManageConfig`), and `src/auto_compact.jl`→`src/auto_manage.jl` (+ legacy + exts +
test files). Pure rename — no behavior change.
The compact *action* is untouched: `compact!`, the `_compact_requested` flag, and
"compaction" wording all stay. Safe to rename: no released/registered version ships
`auto_compact` (latest tag v0.3.7 predates #44/#45/#46; version still 0.3.7).
Validated: renamed CPU + legacy tests green; full Metal suite green (ext rename loads);
runic clean; 0 leftover `auto_compact`/`AUTO_COMPACT`/`AutoCompact` references. CUDA
rename validated on the remote box.
…(CPU) Second action of the auto_manage engine: every `trim_interval`, trim each type's slot tail down to its recent working-set peak, reclaiming the inactive tail that `compact!` only shrinks. Catches the common case of a pool that peaked wide then runs narrow (still entered, so whole-pool-idle schemes never reclaim it). - `_ac_peak_n_active::Int` on TypedPool + BitTypedPool: peak `n_active` since the last trim. One branchless `max` in the claim paths (`_claim_slot!`, `get_bitarray!`), gated by the compile-time `AUTO_MANAGE` const → DCE'd to zero when off (measured ~0.15 ns/acquire, zero-alloc when on). Reset to 0 at each auto-trim, so it doubles as the period accumulator (trim targets the *recent* peak, never the all-time max). - `@atomic _trim_requested::Bool` on AdaptiveArrayPool (+ both ctor paths), parallel to `_compact_requested`. The Timer sets it every `trim_every_ticks = round(trim_interval/interval)` sweeps via a global tick counter. - `_trim_to!(tp, keep)`: generalizes `trim!` (truncate to `keep`, clamped ≥ n_active). - `_run_auto_manage!`: services `_trim_requested` (→ `_auto_trim!`, trim each type to its peak then reset) then `_compact_requested` (→ compact!), each guarded by its flag. The entry hook is unchanged (still triggers on `_compact_requested`, set on the same sweep). `hasfield` guards keep it backend-safe before GPU parity lands. - `enable_auto_manage!(; trim_interval = 120.0)`; `Inf` disables auto-trim. Tests (test/test_auto_trim.jl): peak tracking, `_trim_to!`, auto-trim to the recent peak / to 0 for an unused type, no-op when unflagged, timer cadence, `Inf` disable, zero-alloc hot path, and a RUNTIME_CHECK=1 regression — auto-trim drops the slot reference without swapping the buffer, so an escaped view's poison survives (unlike the compact! poison-undo bug fixed in 988c50f). Tier-1 green.
Mirror the CPU auto-trim data layer (commit 24586d9) onto the Metal backend. The service layer (`_trim_to!`, `_auto_trim!`, `_run_auto_manage!`) is generic over the abstract pool supertypes and already worked for Metal, so this is purely the two telemetry fields plus the claim-path `max`: - `_ac_peak_n_active::Int` on MetalTypedPool: peak `n_active` since the last trim. One branchless `max` in both `_metal_claim_slot!` methods, gated by the compile-time AUTO_MANAGE const -> DCE'd to nothing when off (one Int write when on). - `@atomic _trim_requested::Bool` on MetalAdaptiveArrayPool (+ ctor), parallel to `_compact_requested`. The base-module Timer sweep's `hasfield` guard now const-folds true for Metal pools, so the sweep flags trim every `trim_interval`; the unchanged entry hook (fires on `_compact_requested`, set on the same sweep) lets the generic `_run_auto_manage!` service both at the `@with_pool :metal` safepoint. Tests (test/metal/test_auto_trim.jl): the CPU suite mirrored for Metal -- peak tracking through `_metal_claim_slot!`, `_trim_to!` truncation + regrow, auto-trim to the recent peak / to 0 for an unused type, no-op when unflagged, an R=1 no-buffer-swap guard (the dropped slot's MtlArray DataRef is left intact, unlike compact!), and the sweep flagging `_trim_requested` on a registered Metal pool. Full Metal suite green locally (57s).
Mirror the CPU/Metal auto-trim data layer onto the CUDA backend (commits 24586d9, ab9be38). The service layer (`_trim_to!`, `_auto_trim!`, `_run_auto_manage!`) is generic over the abstract pool supertypes and already worked for CUDA, so this is purely the two telemetry fields plus the claim-path `max`: - `_ac_peak_n_active::Int` on CuTypedPool: peak `n_active` since the last trim. One branchless `max` in both `_cuda_claim_slot!` methods, gated by the compile-time AUTO_MANAGE const -> DCE'd to nothing when off. The operands are host-side `Int` fields (NOT device arrays), so this is a CPU integer op, not a GPU scalar op. - `@atomic _trim_requested::Bool` on CuAdaptiveArrayPool (+ both the main ctor and the `_transfer_cuda_pool` migration ctor, which resets it like `_compact_requested`), parallel to `_compact_requested`. The base-module Timer sweep's `hasfield` guard now const-folds true for CUDA pools, so the sweep flags trim every `trim_interval`; the unchanged entry hook (fires on `_compact_requested`, set on the same sweep) lets the generic `_run_auto_manage!` service both at the `@with_pool :cuda` safepoint. Tests (test/cuda/test_auto_trim.jl): the CPU/Metal suite mirrored for CUDA -- peak tracking through `_cuda_claim_slot!`, `_trim_to!` truncation + regrow, auto-trim to the recent peak / to 0 for an unused type, no-op when unflagged, an S=1 no-buffer-swap guard (the dropped slot's CuArray DataRef is left intact, unlike compact!), and the sweep flagging `_trim_requested` on a registered CUDA pool. Validated locally without a GPU via extension precompile + struct/ctor-arity reflection (fieldcount 7 / 18); the device-level suite runs on the CUDA box.
Close the gap between the unit suites (which test the mechanism in isolation -- set a flag, call the service; the depth-1 safepoint no-op) and the safety guarantee that actually matters: the real background Timer firing repeatedly mid-computation must never corrupt a result. test/test_auto_integration.jl: - A. Differential integrity: a deterministic, pool-heavy simulation (varying array size -> compact bloat; varying concurrency width -> trim tail; nested depth-2 scope) run with auto-manage OFF then ON (real Timer at 0.01s, ~40 sweeps over the run). Asserts the two result vectors are BIT-IDENTICAL -- auto-compact/auto-trim reclaim memory between scopes, never change a computed value. Plus self-heal-after-reclaim correctness, and measurable reclamation (slot tail + byte bloat both shrink). - B. Real-time cadence: over real elapsed seconds, asserts the Timer flags compaction every tick and auto-trim only every K-th tick (K = round(trim_interval/interval)), observed without servicing so the raw cadence is visible. Timing margins match/exceed the existing sleep-based timer tests; verified stable across repeated local runs (9/9 testset, ~5s). CPU only for now; Metal/CUDA mirrors to follow.
Mirror the CPU end-to-end integration suite (006552c) onto Metal and CUDA. Same two guarantees -- (A) the real Timer firing repeatedly mid-computation produces BIT-IDENTICAL results vs auto-manage off, and (B) the compact-every-tick / trim-every-K-th cadence over real elapsed seconds -- adapted for GPU: - No element-wise writes (that is GPU scalar indexing): each slot is fill!ed with a small integer-valued Float32 kept under 2^24 * value, so the on-device `sum` is an exact integer independent of the parallel reduction order (bit-reproducible run-to-run, verified), while fill! over the whole buffer keeps the test sensitive to any aliasing/corruption. Metal: full suite green locally (integration testset 9/9, 4.1s). CUDA: identical mirror, parse-checked locally; the device suite runs on the GPU box.
…amespace) The `_ac_` prefix was a leftover from the pre-rename `auto_compact` era; the field is auto-manage telemetry (consumed by auto-trim), so `_am_` matches AUTO_MANAGE and the `auto_manage` umbrella naming used everywhere else. Pure mechanical rename of the private field across CPU + Metal + CUDA + tests — no public API, branch unreleased. CPU auto suites green; Metal full suite validating.
reshape!(pool, A, dims) with ndims(A) != length(dims) stores its reshaped wrapper in a
SEPARATE placeholder slot while aliasing A's backing memory. _compact_slot! re-pointed
only the wrappers cached AT the compacted slot index, so the cross-slot reshape alias was
left on the freed backing after an active=true compact! -- A followed the new memory while
the reshape view silently diverged (reproduced with default compact! settings on a 1.6 MB
slot, not just contrived parameters).
Fix (all three backends): capture the slot's backing identity BEFORE the swap and re-point
EVERY cached wrapper that shares it, across all slots -- not just wrappers[slot]:
- CPU: match by Memory identity (getfield(w,:ref).mem === old_mem), setfield!(:ref).
- Metal/CUDA: match by DataRef refcount identity (getfield(w,:data).rc === old_rc) and
migrate via the refcount-balanced _update_*_wrapper_data! helper -- the same .data.rc
identity reshape! itself uses to decide buffer sharing.
The scan runs in the cold compaction path (which already allocates a backing + copyto!s),
so the extra work is negligible; every pool wrapper points at offset 0 of its backing, so
Memory/DataRef identity is a precise, offset-safe alias key.
TDD: RED test pinned per backend (test/{,metal/,cuda/}test_compact.jl) -- write through one
view, read the other, plus a shared-backing assertion -- failed before, passes after. CPU
full suite + Metal full suite green; CUDA precompiled + parse-checked (device suite remote).
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #47 +/- ##
==========================================
+ Coverage 95.74% 95.81% +0.07%
==========================================
Files 16 16
Lines 3569 3607 +38
==========================================
+ Hits 3417 3456 +39
+ Misses 152 151 -1
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
This PR introduces a unified background “auto-manage” engine for AdaptiveArrayPools that combines timer-driven capacity compaction with the new working-set-based auto-trim (periodically dropping cold slot tails). It also fixes an active=true compact! correctness issue where reshape! aliases could be left pointing at freed/replaced backing memory.
Changes:
- Rename
auto_compact→auto_manageacross public API, internals, macros, and tests; addtrim_intervalconfiguration and a parallel_trim_requestedflag. - Implement auto-trim: track per-typed-pool peak concurrency (
_am_peak_n_active) on the claim path and periodically truncate slot tails to the recent peak. - Fix
active=true compact!to re-point all cached wrappers that alias a swapped-out backing (CPUMemoryidentity; GPUDataRef.rcidentity), covering cross-slotreshape!placeholders.
Reviewed changes
Copilot reviewed 40 out of 40 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
test/test_compact.jl |
Adds regression test for active=true compact preserving reshape! cross-slot aliasing (CPU). |
test/test_auto_trim.jl |
New CPU unit tests for auto-trim telemetry, _trim_to!, servicing, safety invariants, and cadence. |
test/test_auto_manage.jl |
Renames/updates CPU auto-manage tests (formerly auto-compact). |
test/test_auto_manage_legacy.jl |
Adds legacy (Julia < 1.12) no-op coverage for the renamed auto-manage API. |
test/test_auto_integration.jl |
New CPU end-to-end + real-time timer tests for differential integrity and cadence. |
test/test_auto_compact_legacy.jl |
Removes legacy auto-compact tests in favor of auto-manage legacy tests. |
test/runtests.jl |
Updates suite wiring to include new/renamed auto-manage + auto-trim + integration tests. |
test/metal/test_compact.jl |
Adds Metal regression test for reshape! alias preservation under active=true compact!. |
test/metal/test_auto_trim.jl |
New Metal parity tests for auto-trim behavior and sweep flagging. |
test/metal/test_auto_manage.jl |
Renames/updates Metal auto-manage tests (formerly auto-compact). |
test/metal/test_auto_integration.jl |
New Metal end-to-end + real-time timer tests (integrity + cadence). |
test/metal/runtests.jl |
Wires new/renamed Metal auto-manage + auto-trim + integration tests. |
test/cuda/test_compact.jl |
Adds CUDA regression test for reshape! alias preservation under active=true compact!. |
test/cuda/test_auto_trim.jl |
New CUDA parity tests for auto-trim behavior and sweep flagging. |
test/cuda/test_auto_manage.jl |
Renames/updates CUDA auto-manage tests (formerly auto-compact). |
test/cuda/test_auto_integration.jl |
New CUDA end-to-end + real-time timer tests (integrity + cadence). |
test/cuda/runtests.jl |
Wires new/renamed CUDA auto-manage + auto-trim + integration tests. |
src/types.jl |
Adds _am_peak_n_active telemetry fields and adds _trim_requested to pools (CPU). |
src/task_local_pool.jl |
Updates task-local pool registration to use AUTO_MANAGE / register_auto_manage!. |
src/state.jl |
Adds _trim_to! and fixes wrapper repointing during CPU compaction to handle cross-slot aliases. |
src/macros.jl |
Renames the scope-entry hook emission to _auto_manage_hook / _maybe_auto_manage!. |
src/legacy/auto_manage.jl |
Adds legacy no-op shims for the renamed auto-manage API (Julia < 1.12). |
src/legacy/auto_compact.jl |
Removes legacy auto-compact shim in favor of legacy auto-manage shim. |
src/bitarray.jl |
Adds peak-concurrency tracking for BitTypedPool acquisition when AUTO_MANAGE is enabled. |
src/auto_manage.jl |
New unified engine: registry sweep, trim cadence, servicing logic, timer lifecycle, public API. |
src/auto_compact.jl |
Removes old auto-compact engine implementation. |
src/AdaptiveArrayPools.jl |
Exports and includes updated auto-manage module and legacy path wiring. |
src/acquire.jl |
Adds hot-path peak tracking (_am_peak_n_active = max(...)) for typed-pool slot claims. |
ext/AdaptiveArrayPoolsMetalExt/types.jl |
Adds _am_peak_n_active + _trim_requested fields for Metal typed/pool types. |
ext/AdaptiveArrayPoolsMetalExt/task_local_pool.jl |
Updates Metal pool registration to use auto-manage registry. |
ext/AdaptiveArrayPoolsMetalExt/state.jl |
Fixes Metal compaction wrapper repointing to handle cross-slot buffer aliases. |
ext/AdaptiveArrayPoolsMetalExt/auto_manage.jl |
Renames and wires Metal scope-entry servicing hook to _run_auto_manage!. |
ext/AdaptiveArrayPoolsMetalExt/AdaptiveArrayPoolsMetalExt.jl |
Updates Metal extension includes to use auto_manage.jl. |
ext/AdaptiveArrayPoolsMetalExt/acquire.jl |
Adds Metal hot-path peak tracking for auto-trim telemetry. |
ext/AdaptiveArrayPoolsCUDAExt/types.jl |
Adds _am_peak_n_active + _trim_requested fields for CUDA typed/pool types. |
ext/AdaptiveArrayPoolsCUDAExt/task_local_pool.jl |
Updates CUDA pool registration to use auto-manage registry. |
ext/AdaptiveArrayPoolsCUDAExt/state.jl |
Fixes CUDA compaction wrapper repointing to handle cross-slot buffer aliases. |
ext/AdaptiveArrayPoolsCUDAExt/auto_manage.jl |
Renames and wires CUDA scope-entry servicing hook to _run_auto_manage!. |
ext/AdaptiveArrayPoolsCUDAExt/AdaptiveArrayPoolsCUDAExt.jl |
Updates CUDA extension includes to use auto_manage.jl. |
ext/AdaptiveArrayPoolsCUDAExt/acquire.jl |
Adds CUDA hot-path peak tracking for auto-trim telemetry. |
Comments suppressed due to low confidence (3)
test/test_auto_manage.jl:246
- Typo in comment: “auto-manageed” should be “auto-managed”.
test/metal/test_auto_manage.jl:130 - Typo in comment: “auto-manageed” should be “auto-managed”.
test/cuda/test_auto_manage.jl:130 - Typo in comment: “auto-manageed” should be “auto-managed”.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…nfig) Add a Features page covering the background auto-compact/auto-trim engine — what it reclaims, the @with_pool safepoint model (why it can't change results), default-on, the enable/disable knobs, the zero-alloc trade-off + auto_manage=false escape hatch, and manual compact!/trim!. Add compact! + the auto_manage symbols to the API reference tables and an auto_manage preference section to Configuration. Also fix a pre-existing broken [Safety](safety.md) link in configuration.md (→ ../safety/overview.md).
Review fixes (GitHub Copilot): - Legacy (Julia <1.12) `enable_auto_manage!` shim now accepts (and ignores) the new `trim_interval` keyword, so the public API is call-compatible across all supported Julia (previously a MethodError on old Julia when passing trim_interval). - Refresh the `enable_auto_manage!` docstring: show `trim_interval`, document auto-trim, generalize the "what gets managed" note (compaction + trim), drop the typo. - Fix rename artifacts from the auto_compact->auto_manage token replace: "auto-manageion" -> "auto-management" / "auto-manageed" -> "auto-managed" (15 comments across 10 files), and the misleading "Auto-Compact" file/section headers -> "Auto-Manage". Patch coverage (was 94.07% on src/auto_manage.jl, the only file below 100%): - Cover the `_auto_trim!` `others`-type loop with a non-fixed-slot (UInt8) auto-trim test. - Cover the two `_run_auto_manage!` try/catch+@warn defensive branches by parking a non-pool object in both others collections (values(others) for trim, _others_values for CPU compact) — mirrors the existing throwing-sweep test. src/auto_manage.jl now 100% (90/90).
Review fixes (GitHub Copilot): - Legacy (Julia <1.12) `enable_auto_manage!` shim now accepts (and ignores) the new `trim_interval` keyword, so the public API is call-compatible across all supported Julia (previously a MethodError on old Julia when passing trim_interval). - Refresh the `enable_auto_manage!` docstring: show `trim_interval`, document auto-trim, generalize the "what gets managed" note (compaction + trim), drop the typo. - Fix rename artifacts from the auto_compact->auto_manage token replace: "auto-manageion" -> "auto-management" / "auto-manageed" -> "auto-managed" (15 comments across 10 files), and the misleading "Auto-Compact" file/section headers -> "Auto-Manage". Patch coverage (was 94.07% on src/auto_manage.jl, the only file below 100%): - Cover the `_auto_trim!` `others`-type loop with a non-fixed-slot (UInt8) auto-trim test. - Cover the two `_run_auto_manage!` try/catch+@warn defensive branches by parking a non-pool object in both others collections (values(others) for trim, _others_values for CPU compact) — mirrors the existing throwing-sweep test. src/auto_manage.jl now 100% (90/90).
1a51f6f to
1b5fd3f
Compare
Polish the public tuning API before it ships in v0.3.8: - Rename `enable_auto_manage!` keywords to self-documenting names: `interval` -> `compact_interval`, `factor` -> `compact_bloat_factor` (compact a slot at >= this x its live size), `shrink_to` -> `compact_target_ratio` (shrink to this x live size), `min_bytes` -> `compact_min_bytes`. `trim_interval` kept. - Drop the `active` keyword: auto-manage only ever runs at the depth-1 safepoint where `n_active == 0`, so compacting "active" slots is a no-op concept here (it was a `compact!`-only knob leaking into the auto API). `compact!`'s own kwargs are unchanged. - The keyword defaults now come from flat `auto_manage_*` Preferences (`auto_manage_compact_interval`, `auto_manage_trim_interval`, `auto_manage_compact_bloat_factor`, `auto_manage_compact_target_ratio`, `auto_manage_compact_min_bytes`), read once at load and coerced (`30` or `30.0` both fine). So the common case is to set them once in `LocalPreferences.toml`, no startup code; the keywords are the runtime override. `_AutoManageConfig` fields renamed to match. Updates the legacy (<1.12) shim signature, all CPU/Metal/CUDA tests, the docstring, and the Automatic Memory Management + Configuration docs. CPU auto suites green.
Add an ASCII before/after diagram in "What it does" making the two actions visual: auto-compact shrinks the bloated buffer *within* one slot (a bar that gets shorter), auto-trim drops *whole* slots beyond the recent peak (boxes that disappear). Renders in Documenter's monospace code block — no Mermaid/plugin dependency. Also fixes the remaining old kwarg names on this page (interval→compact_interval, factor→compact_bloat_factor, min_bytes→compact_min_bytes) left over from the config rename.
Reframe "What it does": present `compact!` and `trim!` as the two primitives (with the before/after schematic), then `auto_manage` as "run those automatically on a timer, a bit smartly." The trim diagram now shows the ACTIVE slots (█ in use vs ░ idle-retained) so it's clear trim keeps what's in use and drops the idle tail. The auto-* table adds the smart part: auto-compact only touches bloated slots; auto-trim keeps the recent working-set PEAK (not the instantaneous count) so a brief widening isn't thrashed.
Explicit release anchor for `@JuliaRegistrator register`. Project.toml is already at 0.3.8 (set in e7126ba); this dedicated empty commit is a clear target for the registration trigger comment.
Summary
Adds auto-trim and unifies it with auto-compact under one renamed engine,
auto_compact→auto_manage.Auto-compact shrinks a bloated slot's backing; auto-trim drops the slot tail itself — every
trim_interval, each type's slots are trimmed to the recent working-set peak (a type unused for the period trims to 0). OneTimerdrives both:compact!every tick, auto-trim everytrim_interval.How it works
Each
TypedPooltracks the peakn_activesince the last trim in_am_peak_n_active— one branchlessmaxin the claim path, gated by the compile-timeAUTO_MANAGE. The Timer sweep sets@atomic _compact_requestedevery tick and@atomic _trim_requestedeveryround(trim_interval/interval)ticks; the owner task services both at its@with_pooldepth-1 safepoint via the shared_run_auto_manage!(trim first, then compact)._trim_to!truncatesvectors/arr_wrappers/slot_extentsto the peak, dropping references only — no buffer swap, soRUNTIME_CHECKpoison on a dropped slot survives.API
trim_intervalis the new knob;Infdisables auto-trim (compact-only). The rename covers the whole unreleased auto-compact surface (const/Preference/API/internals/files/tests);_compact_requestedis unchanged,_trim_requestedis its parallel.Zero-overhead
One host-side
Intmaxper acquire, gated byAUTO_MANAGE: DCE'd to nothing when off, ~0.15 ns when on, zero-alloc. On GPU it is not a device/scalar op (operands are hostInts) — verified clean underMetal.allowscalar(false), ~0.37 % of one Metalacquire!.Also fixed:
active=true compact!strandedreshape!aliasesreshape!(pool, A, dims)(ndims differ) caches its wrapper in a separate placeholder slot while aliasingA's backing._compact_slot!re-pointed only the wrappers at the compacted slot, so anactive=true compact!left the cross-slot alias on freed memory (silent divergence, reproduced with default settings). Fix (all backends): re-point every cached wrapper sharing the swapped-out backing, matched by identity — CPUMemory(:ref.mem), GPUDataRef(:data.rc). Cold path; negligible cost.Tested · backends
CPU + Metal + CUDA, Julia 1.12+ (≤1.11: renamed no-op shim). Auto-trim: peak tracking,
_trim_to!, service to recent peak / to 0, zero-alloc,S=1poison-preservation, Timer cadence (Infdisables). End-to-end: a differential-integrity test asserts bit-identical results with the real Timer firing mid-computation (the "never corrupts a simulation" contract) + a real-time compact/trim-cadence test. Plus the per-backend reshape-alias regression.CPU (latest + LTS) and Metal full suites green; CUDA precompiled + parse-checked, device suite runs on the GPU box.
Out of scope
Auto-empty (needs cooperative release; dead-task pools are GC'd via
WeakRef) · idle-stats escalation · Timer-thread direct trimming.