Repository navigation
(feat): trim! — manual reclamation of inactive pool slots - #44
Merged
Merged
Conversation
…tal) Add `trim!(pool; force_gc=false)` to release backing buffers and cached wrappers for INACTIVE retained slots (n_active+1:end) while keeping ACTIVE slots intact. Fills the gap between `reset!` (keeps all memory) and `empty!` (drops everything). - Generic core on AbstractTypedPool: truncates `vectors` AND `arr_wrappers` to n_active. Cached wrappers pin backing buffers via :ref, so both must be dropped for the memory to become GC-eligible. - Capacity-aware byte estimate: `summarysize` (CPU), `sizeof` (Metal GPU). - Variants: per-type `trim!(pool, T)`, zero-arg `trim!()` (task-local pool), `DisabledPool` no-op. Returns `(; slots_released, wrappers_released, estimated_bytes_released, gc_triggered)`. - Metal parity (CuTypedPool deferred). Legacy (Julia <1.12) defines `trim!` as an exported no-op (zero summary + one-time @warn) so dependents stay portable across the full [compat] julia range. - Tests pin: active slots untouched (data + identity), inactive dropped, wrapper-cache truncation, byte accounting (S=0/S=1), self-heal, Bit/fallback. Verified on Julia 1.12 (CPU + Metal) and 1.10 LTS (legacy no-op). Docs: document `trim!` and correct a prior inaccuracy (`reset!` preserves buffers; it does not release memory).
Mirror the Metal `trim!` into the CUDA extension: - `trim!(pool::CuAdaptiveArrayPool; force_gc=false)` + per-type variant, reusing the generic `_trim_inactive_typed_pool!` core (CuTypedPool shares the CPU/Metal field layout, so the helpers apply unchanged). - `_inactive_storage_bytes(::CuTypedPool, …)` override using `sizeof` for true GPU buffer bytes (summarysize only sees the CPU-side handle). - `test/cuda/test_trim.jl` mirrors the Metal suite (active-untouched data+identity, byte accounting, wrapper-cache truncation, per-type, force_gc, self-heal); runs on CI with a functional CUDA device, skipped otherwise. Validated locally by extension precompilation (dispatch resolves); GPU execution deferred to CI. Spec/API docs updated to reflect CPU + Metal + CUDA support.
Address code-review findings: - [P2] GPU `_inactive_storage_bytes` used `sizeof`, which follows logical length. In runtime-check mode (R/S>=1), rewind shrinks released slots to dims 0 via `_resize_to_fit!` while preserving `maxsize`, so the estimate read 0 for buffers still holding device capacity. Use `getfield(v, :maxsize)` (allocated device bytes) on Metal + CUDA, and add R=1/S=1 safety-mode byte-accounting tests. - [P3] api-essentials wording implied `trim!` preserves "warm, normal-sized" buffers, but it drops ALL inactive retained slots (at top level, every reusable buffer). Reword to the actual guarantee: only active arrays are preserved. Verified: Metal suite green on Apple Silicon (incl. the new R=1 test); CUDA ext precompiles and `trim!` dispatch confirmed (GPU execution on CI). CPU unaffected (`summarysize` is already capacity-aware).
…f copy) The MtlArray wrapper-creation sites passed `copy(getfield(vec, :data))` to the MtlArray constructor, but that constructor already copies/retains its DataRef argument. The extra copy created a second retained DataRef that was never released, pinning each buffer's refcount above 0 — so the GPU buffer was never freed by trim!, empty!, GC, or even explicit unsafe_free!. Pass the DataRef directly (the constructor copies it) at both wrapper sites (get_array! cache-miss and reshape cache-miss). Buffer sharing is unchanged (write-through verified); the refcount now reaches 0 and Metal device memory is reclaimed. Verified on Apple Silicon: 4 GiB -> 384 KiB after trim!(force_gc=true); full Metal suite green incl. a new regression test asserting device memory drops. CUDA is unaffected: its CuArray constructor takes ownership of the DataRef (no internal copy), so CUDA's explicit copy() is required and correct.
…ripts
- CUDA: force_gc=true now also calls CUDA.reclaim() to return pooled blocks to
the driver (Metal/CPU have no further reclaim step). Docstring updated to
describe the backend-aware behavior.
- Add scripts/trim_smoke_{metal,cuda}.jl: manual scripts that print the Julia
heap (gc_live_bytes), process RSS, and GPU device memory before/after trim!
to confirm real reclamation. The Metal script runs on Apple Silicon; the CUDA
script on an NVIDIA box.
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #44 +/- ##
==========================================
+ Coverage 95.61% 95.63% +0.01%
==========================================
Files 14 14
Lines 3309 3391 +82
==========================================
+ Hits 3164 3243 +79
- Misses 145 148 +3
🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
Pull request overview
Adds a new manual reclamation API, trim!, to release pool references to inactive retained slots (including cached wrappers that would otherwise pin buffers), while keeping the active working set intact. This complements existing reset! (mark inactive but keep buffers) and empty! (drop everything) across CPU + Metal + CUDA, with a legacy (Julia < 1.12) no-op implementation for API stability.
Changes:
- Implement
trim!in the core pool, plus backend-specific overrides for GPU byte accounting and (CUDA-only)CUDA.reclaim()integration. - Fix Metal wrapper construction to avoid an extra
copy()ofDataRefthat could keep GPU buffers pinned and prevent reclamation. - Add comprehensive CPU/legacy/Metal/CUDA tests, update docs, and include manual smoke-test scripts.
Reviewed changes
Copilot reviewed 18 out of 18 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| test/test_trim.jl | CPU test coverage for trim! semantics (active preserved, inactive dropped, wrapper-cache trimming, bytes estimate, self-heal). |
| test/test_trim_legacy.jl | Legacy (Julia < 1.12) no-op behavior tests (zero summary + warn-once). |
| test/runtests.jl | Wires new CPU/legacy trim tests into the main test runner. |
| test/metal/test_trim.jl | Metal parity tests, including device-memory reclamation regression coverage. |
| test/metal/runtests.jl | Wires Metal trim tests into the Metal test runner. |
| test/cuda/test_trim.jl | CUDA parity tests for trimming behavior and byte accounting. |
| test/cuda/runtests.jl | Wires CUDA trim tests into the CUDA test runner. |
| src/state.jl | Core trim! implementation + helpers for trimming inactive vectors and wrapper caches; DisabledPool methods. |
| src/legacy/state.jl | Legacy trim! no-op implementation (warn-once) for Julia < 1.12. |
| src/AdaptiveArrayPools.jl | Exports trim! from the public API. |
| scripts/trim_smoke_metal.jl | Manual Metal memory smoke-test script for before/after trim reporting. |
| scripts/trim_smoke_cuda.jl | Manual CUDA memory smoke-test script including CUDA.memory_status() output. |
| ext/AdaptiveArrayPoolsMetalExt/state.jl | Metal-specific trim! entrypoints + GPU-capacity byte accounting via maxsize. |
| ext/AdaptiveArrayPoolsMetalExt/acquire.jl | Metal fix: avoid double-copying DataRef when constructing cached wrappers. |
| ext/AdaptiveArrayPoolsCUDAExt/state.jl | CUDA-specific trim! entrypoints + maxsize accounting + CUDA.reclaim() on force_gc. |
| docs/src/reference/api.md | API reference updates for reset! / trim! / empty!. |
| docs/src/basics/api-essentials.md | User-facing docs clarifying reset! vs trim! vs empty! semantics and returns. |
| docs/design/selective_memory_reclamation.md | Design/implementation rationale and semantics for inactive-slot trimming. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
…lot review, PR #44) Per-type `trim!` called `get_typed_pool!(pool, T)`, which registers a new fallback pool in `pool.others` for a never-used type — a surprising mutation/allocation for a reclamation API. It now no-ops (zero summary, still honoring `force_gc`) unless `T` is a fixed-slot type or already present in `others`. Applied to CPU + Metal + CUDA via a per-backend fixed-slot type Union; fixed-slot access is unaffected. Also fills the trim! coverage gaps codecov flagged on PR #44: - modern: per-type `force_gc` path and the new no-create path (test/test_trim.jl) - legacy (Julia <1.12): `DisabledPool` trim! no-ops (test/test_trim_legacy.jl) Verified: full CPU+Metal suite green (1.12); legacy test green (1.10); the trim! region of src/state.jl is fully covered (guard branch + per-type force_gc hit).
Type stability: trim!(pool) inferred its summary as
NamedTuple{...,<:Tuple{Any,Any,Any,Bool}} — the foreach_fixed_slot closure boxed
the accumulators (Core.Box) and the others-loop over Vector{Any} widened the
count fields to Any. Replaced with an unrolled @generated fold
(_trim_fixed_counts!) plus a barrier loop (_trim_others_counts!), and funneled
every overload through one _trim_summary helper. The return is now the concrete
@NamedTuple{...::Int, ..., gc_triggered::Bool} (also drops ~288B of boxing).
Varargs parity: added trim!(pool, types::Type...) on CPU, CUDA, Metal, legacy,
and DisabledPool, mirroring checkpoint!/rewind!/reset!. A shared
_trim_one_counts! guard (skips never-used fallback types) backs both the
single-type and varargs forms; GPU reuses the type-stable Ref-accumulator path.
Tests: type-stability (@inferred + return_types for all positional forms),
varargs behavior + per-type guard, N-D wrapper-cache rebuild after trim, and
mid-scope trim leaving the checkpoint stack untouched before a later rewind.
GPU varargs tests added to the CUDA/Metal suites.
Comments: fixed the stale Metal get_array! docstring (DataRef passed directly,
not copy()), softened the CUDA force_gc reclaim wording to match the docstring's
"not guaranteed" hedge, and dropped PR-ref provenance from CUDA/Metal source.
mgyoo86
added a commit
that referenced
this pull request
Jul 1, 2026
* refactor: rename auto_compact → auto_manage (unify the maintenance engine)
The background-maintenance engine is about to gain a second action (auto-trim), so
"compact" undersells it. Rename the unreleased umbrella `auto_compact` → `auto_manage`
across CPU, legacy, Metal, CUDA, and tests — const `AUTO_COMPACT`→`AUTO_MANAGE`,
Preference `auto_compact`→`auto_manage`, `enable_/disable_/…_enabled`, the engine
internals (`_run_/_maybe_/register_/_…_sweep!`, `_AUTO_MANAGE_{REGISTRY,TIMER,CONFIG,LOCK}`,
`_AutoManageConfig`), and `src/auto_compact.jl`→`src/auto_manage.jl` (+ legacy + exts +
test files). Pure rename — no behavior change.
The compact *action* is untouched: `compact!`, the `_compact_requested` flag, and
"compaction" wording all stay. Safe to rename: no released/registered version ships
`auto_compact` (latest tag v0.3.7 predates #44/#45/#46; version still 0.3.7).
Validated: renamed CPU + legacy tests green; full Metal suite green (ext rename loads);
runic clean; 0 leftover `auto_compact`/`AUTO_COMPACT`/`AutoCompact` references. CUDA
rename validated on the remote box.
* feat(auto_manage): auto-trim — periodic working-set tail reclamation (CPU)
Second action of the auto_manage engine: every `trim_interval`, trim each type's
slot tail down to its recent working-set peak, reclaiming the inactive tail that
`compact!` only shrinks. Catches the common case of a pool that peaked wide then
runs narrow (still entered, so whole-pool-idle schemes never reclaim it).
- `_ac_peak_n_active::Int` on TypedPool + BitTypedPool: peak `n_active` since the
last trim. One branchless `max` in the claim paths (`_claim_slot!`, `get_bitarray!`),
gated by the compile-time `AUTO_MANAGE` const → DCE'd to zero when off (measured
~0.15 ns/acquire, zero-alloc when on). Reset to 0 at each auto-trim, so it doubles
as the period accumulator (trim targets the *recent* peak, never the all-time max).
- `@atomic _trim_requested::Bool` on AdaptiveArrayPool (+ both ctor paths), parallel
to `_compact_requested`. The Timer sets it every `trim_every_ticks =
round(trim_interval/interval)` sweeps via a global tick counter.
- `_trim_to!(tp, keep)`: generalizes `trim!` (truncate to `keep`, clamped ≥ n_active).
- `_run_auto_manage!`: services `_trim_requested` (→ `_auto_trim!`, trim each type to
its peak then reset) then `_compact_requested` (→ compact!), each guarded by its flag.
The entry hook is unchanged (still triggers on `_compact_requested`, set on the same
sweep). `hasfield` guards keep it backend-safe before GPU parity lands.
- `enable_auto_manage!(; trim_interval = 120.0)`; `Inf` disables auto-trim.
Tests (test/test_auto_trim.jl): peak tracking, `_trim_to!`, auto-trim to the recent
peak / to 0 for an unused type, no-op when unflagged, timer cadence, `Inf` disable,
zero-alloc hot path, and a RUNTIME_CHECK=1 regression — auto-trim drops the slot
reference without swapping the buffer, so an escaped view's poison survives (unlike
the compact! poison-undo bug fixed in 988c50f). Tier-1 green.
* feat(auto_manage): auto-trim Metal parity
Mirror the CPU auto-trim data layer (commit 24586d9) onto the Metal backend. The
service layer (`_trim_to!`, `_auto_trim!`, `_run_auto_manage!`) is generic over the
abstract pool supertypes and already worked for Metal, so this is purely the two
telemetry fields plus the claim-path `max`:
- `_ac_peak_n_active::Int` on MetalTypedPool: peak `n_active` since the last trim.
One branchless `max` in both `_metal_claim_slot!` methods, gated by the compile-time
AUTO_MANAGE const -> DCE'd to nothing when off (one Int write when on).
- `@atomic _trim_requested::Bool` on MetalAdaptiveArrayPool (+ ctor), parallel to
`_compact_requested`. The base-module Timer sweep's `hasfield` guard now const-folds
true for Metal pools, so the sweep flags trim every `trim_interval`; the unchanged
entry hook (fires on `_compact_requested`, set on the same sweep) lets the generic
`_run_auto_manage!` service both at the `@with_pool :metal` safepoint.
Tests (test/metal/test_auto_trim.jl): the CPU suite mirrored for Metal -- peak
tracking through `_metal_claim_slot!`, `_trim_to!` truncation + regrow, auto-trim to
the recent peak / to 0 for an unused type, no-op when unflagged, an R=1 no-buffer-swap
guard (the dropped slot's MtlArray DataRef is left intact, unlike compact!), and the
sweep flagging `_trim_requested` on a registered Metal pool. Full Metal suite green
locally (57s).
* feat(auto_manage): auto-trim CUDA parity
Mirror the CPU/Metal auto-trim data layer onto the CUDA backend (commits 24586d9,
ab9be38). The service layer (`_trim_to!`, `_auto_trim!`, `_run_auto_manage!`) is
generic over the abstract pool supertypes and already worked for CUDA, so this is
purely the two telemetry fields plus the claim-path `max`:
- `_ac_peak_n_active::Int` on CuTypedPool: peak `n_active` since the last trim. One
branchless `max` in both `_cuda_claim_slot!` methods, gated by the compile-time
AUTO_MANAGE const -> DCE'd to nothing when off. The operands are host-side `Int`
fields (NOT device arrays), so this is a CPU integer op, not a GPU scalar op.
- `@atomic _trim_requested::Bool` on CuAdaptiveArrayPool (+ both the main ctor and
the `_transfer_cuda_pool` migration ctor, which resets it like `_compact_requested`),
parallel to `_compact_requested`. The base-module Timer sweep's `hasfield` guard now
const-folds true for CUDA pools, so the sweep flags trim every `trim_interval`; the
unchanged entry hook (fires on `_compact_requested`, set on the same sweep) lets the
generic `_run_auto_manage!` service both at the `@with_pool :cuda` safepoint.
Tests (test/cuda/test_auto_trim.jl): the CPU/Metal suite mirrored for CUDA -- peak
tracking through `_cuda_claim_slot!`, `_trim_to!` truncation + regrow, auto-trim to
the recent peak / to 0 for an unused type, no-op when unflagged, an S=1 no-buffer-swap
guard (the dropped slot's CuArray DataRef is left intact, unlike compact!), and the
sweep flagging `_trim_requested` on a registered CUDA pool. Validated locally without a
GPU via extension precompile + struct/ctor-arity reflection (fieldcount 7 / 18); the
device-level suite runs on the CUDA box.
* test(auto_manage): end-to-end integrity + real-time cadence
Close the gap between the unit suites (which test the mechanism in isolation -- set a
flag, call the service; the depth-1 safepoint no-op) and the safety guarantee that
actually matters: the real background Timer firing repeatedly mid-computation must
never corrupt a result.
test/test_auto_integration.jl:
- A. Differential integrity: a deterministic, pool-heavy simulation (varying array
size -> compact bloat; varying concurrency width -> trim tail; nested depth-2 scope)
run with auto-manage OFF then ON (real Timer at 0.01s, ~40 sweeps over the run).
Asserts the two result vectors are BIT-IDENTICAL -- auto-compact/auto-trim reclaim
memory between scopes, never change a computed value. Plus self-heal-after-reclaim
correctness, and measurable reclamation (slot tail + byte bloat both shrink).
- B. Real-time cadence: over real elapsed seconds, asserts the Timer flags compaction
every tick and auto-trim only every K-th tick (K = round(trim_interval/interval)),
observed without servicing so the raw cadence is visible.
Timing margins match/exceed the existing sleep-based timer tests; verified stable
across repeated local runs (9/9 testset, ~5s). CPU only for now; Metal/CUDA mirrors
to follow.
* test(auto_manage): GPU integrity + real-time cadence (Metal + CUDA)
Mirror the CPU end-to-end integration suite (006552c) onto Metal and CUDA. Same two
guarantees -- (A) the real Timer firing repeatedly mid-computation produces BIT-IDENTICAL
results vs auto-manage off, and (B) the compact-every-tick / trim-every-K-th cadence over
real elapsed seconds -- adapted for GPU:
- No element-wise writes (that is GPU scalar indexing): each slot is fill!ed with a small
integer-valued Float32 kept under 2^24 * value, so the on-device `sum` is an exact integer
independent of the parallel reduction order (bit-reproducible run-to-run, verified),
while fill! over the whole buffer keeps the test sensitive to any aliasing/corruption.
Metal: full suite green locally (integration testset 9/9, 4.1s). CUDA: identical mirror,
parse-checked locally; the device suite runs on the GPU box.
* refactor: rename _ac_peak_n_active → _am_peak_n_active (auto_manage namespace)
The `_ac_` prefix was a leftover from the pre-rename `auto_compact` era; the field is
auto-manage telemetry (consumed by auto-trim), so `_am_` matches AUTO_MANAGE and the
`auto_manage` umbrella naming used everywhere else. Pure mechanical rename of the private
field across CPU + Metal + CUDA + tests — no public API, branch unreleased. CPU auto
suites green; Metal full suite validating.
* fix(compact): active compact! no longer strands reshape! aliases
reshape!(pool, A, dims) with ndims(A) != length(dims) stores its reshaped wrapper in a
SEPARATE placeholder slot while aliasing A's backing memory. _compact_slot! re-pointed
only the wrappers cached AT the compacted slot index, so the cross-slot reshape alias was
left on the freed backing after an active=true compact! -- A followed the new memory while
the reshape view silently diverged (reproduced with default compact! settings on a 1.6 MB
slot, not just contrived parameters).
Fix (all three backends): capture the slot's backing identity BEFORE the swap and re-point
EVERY cached wrapper that shares it, across all slots -- not just wrappers[slot]:
- CPU: match by Memory identity (getfield(w,:ref).mem === old_mem), setfield!(:ref).
- Metal/CUDA: match by DataRef refcount identity (getfield(w,:data).rc === old_rc) and
migrate via the refcount-balanced _update_*_wrapper_data! helper -- the same .data.rc
identity reshape! itself uses to decide buffer sharing.
The scan runs in the cold compaction path (which already allocates a backing + copyto!s),
so the extra work is negligible; every pool wrapper points at offset 0 of its backing, so
Memory/DataRef identity is a precise, offset-safe alias key.
TDD: RED test pinned per backend (test/{,metal/,cuda/}test_compact.jl) -- write through one
view, read the other, plus a shared-backing assertion -- failed before, passes after. CPU
full suite + Metal full suite green; CUDA precompiled + parse-checked (device suite remote).
* docs: document auto_manage (Automatic Memory Management page + API/config)
Add a Features page covering the background auto-compact/auto-trim engine — what it
reclaims, the @with_pool safepoint model (why it can't change results), default-on, the
enable/disable knobs, the zero-alloc trade-off + auto_manage=false escape hatch, and
manual compact!/trim!. Add compact! + the auto_manage symbols to the API reference
tables and an auto_manage preference section to Configuration. Also fix a pre-existing
broken [Safety](safety.md) link in configuration.md (→ ../safety/overview.md).
* fix(auto_manage): address PR #47 review + raise patch coverage to 100%
Review fixes (GitHub Copilot):
- Legacy (Julia <1.12) `enable_auto_manage!` shim now accepts (and ignores) the new
`trim_interval` keyword, so the public API is call-compatible across all supported Julia
(previously a MethodError on old Julia when passing trim_interval).
- Refresh the `enable_auto_manage!` docstring: show `trim_interval`, document auto-trim,
generalize the "what gets managed" note (compaction + trim), drop the typo.
- Fix rename artifacts from the auto_compact->auto_manage token replace: "auto-manageion"
-> "auto-management" / "auto-manageed" -> "auto-managed" (15 comments across 10 files),
and the misleading "Auto-Compact" file/section headers -> "Auto-Manage".
Patch coverage (was 94.07% on src/auto_manage.jl, the only file below 100%):
- Cover the `_auto_trim!` `others`-type loop with a non-fixed-slot (UInt8) auto-trim test.
- Cover the two `_run_auto_manage!` try/catch+@warn defensive branches by parking a non-pool
object in both others collections (values(others) for trim, _others_values for CPU compact)
— mirrors the existing throwing-sweep test. src/auto_manage.jl now 100% (90/90).
* (release): bump version to v0.3.8
* feat(auto_manage): descriptive config kwargs + LocalPreferences defaults
Polish the public tuning API before it ships in v0.3.8:
- Rename `enable_auto_manage!` keywords to self-documenting names:
`interval` -> `compact_interval`, `factor` -> `compact_bloat_factor`
(compact a slot at >= this x its live size), `shrink_to` -> `compact_target_ratio`
(shrink to this x live size), `min_bytes` -> `compact_min_bytes`. `trim_interval` kept.
- Drop the `active` keyword: auto-manage only ever runs at the depth-1 safepoint where
`n_active == 0`, so compacting "active" slots is a no-op concept here (it was a
`compact!`-only knob leaking into the auto API). `compact!`'s own kwargs are unchanged.
- The keyword defaults now come from flat `auto_manage_*` Preferences
(`auto_manage_compact_interval`, `auto_manage_trim_interval`,
`auto_manage_compact_bloat_factor`, `auto_manage_compact_target_ratio`,
`auto_manage_compact_min_bytes`), read once at load and coerced (`30` or `30.0` both
fine). So the common case is to set them once in `LocalPreferences.toml`, no startup
code; the keywords are the runtime override. `_AutoManageConfig` fields renamed to match.
Updates the legacy (<1.12) shim signature, all CPU/Metal/CUDA tests, the docstring, and
the Automatic Memory Management + Configuration docs. CPU auto suites green.
* test(auto_manage): cover the auto_manage_* Preference defaults
* docs(auto-manage): add a before/after schematic for compact vs trim
Add an ASCII before/after diagram in "What it does" making the two actions visual:
auto-compact shrinks the bloated buffer *within* one slot (a bar that gets shorter),
auto-trim drops *whole* slots beyond the recent peak (boxes that disappear). Renders in
Documenter's monospace code block — no Mermaid/plugin dependency. Also fixes the remaining
old kwarg names on this page (interval→compact_interval, factor→compact_bloat_factor,
min_bytes→compact_min_bytes) left over from the config rename.
* docs(auto-manage): define compact/trim first, then auto-* as timed+smart
Reframe "What it does": present `compact!` and `trim!` as the two primitives (with the
before/after schematic), then `auto_manage` as "run those automatically on a timer, a bit
smartly." The trim diagram now shows the ACTIVE slots (█ in use vs ░ idle-retained) so it's
clear trim keeps what's in use and drops the idle tail. The auto-* table adds the smart
part: auto-compact only touches bloated slots; auto-trim keeps the recent working-set PEAK
(not the instantaneous count) so a brief widening isn't thrashed.
mgyoo86
referenced
this pull request
Jul 1, 2026
Explicit release anchor for `@JuliaRegistrator register`. Project.toml is already at 0.3.8 (set in e7126ba); this dedicated empty commit is a clear target for the registration trigger comment.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds
trim!, a manual operation that releases the pool's references to inactive retained slots while keeping the active working set intact.It fills the gap between
reset!(keeps all memory) andempty!(drops everything):trim!drops only the backing buffers and cached wrappers that are held purely for reuse, so they become GC-eligible.API
force_gc=truerunsGC.gc(); on CUDA it additionally callsCUDA.reclaim()to return pooled blocks to the driver.Semantics
Each typed pool splits into active slots (
1:n_active, in use by the live call stack) and inactive retained slots (n_active+1:end, kept only as a reuse cache).trim!truncates bothvectorsand the wrapper caches ton_active— the wrapper cache must go too, since a cached wrapper pins its backing buffer.Active slots are never touched, and no replacement buffers are allocated (the cold
acquire!path re-allocates a trimmed slot if it is needed again).estimated_bytes_releasedis capacity-aware (summarysizeon CPU,maxsizeon GPU), so it counts buffers even after runtime-check mode has shrunk them to length 0.Backends & versions
CPU + Metal + CUDA, on Julia 1.12+.
On Julia ≤1.11 (the legacy N-way-cache path)
trim!is a defined, exported no-op (zero summary + one-time@warn), so dependent packages can call it across the full[compat] juliarange without anUndefVarError.Included fix: Metal GPU-memory was never reclaimed
Verifying reclamation surfaced a pre-existing Metal bug.
The wrapper-creation sites passed
copy(getfield(vec, :data))to theMtlArrayconstructor, but that constructor already copies itsDataRef— the extra copy left an orphaned retainedDataRefthat pinned the buffer's refcount above 0, so the GPU buffer was never freed bytrim!orempty!/reset!.Fixed by passing the
DataRefdirectly at both sites; buffer sharing is unchanged (write-through verified).Verified on Apple Silicon: 4 GiB → 384 KiB after
trim!(force_gc=true), guarded by a regression test.CUDA is unaffected — its
CuArrayconstructor takes ownership of theDataRef(no internal copy), so CUDA's explicitcopy()is required and correct.Tested
Active-untouched (data + object identity), inactive dropped, wrapper-cache truncation, byte accounting (S=0 and S=1), self-heal after trim,
Bit/fallback types, the per-type / zero-arg /DisabledPoolforms, and a Metal device-memory-drop regression test.Files:
test/test_trim.jl(CPU) ·test/test_trim_legacy.jl(≤1.11 no-op) ·test/metal/test_trim.jl·test/cuda/test_trim.jl.Manual GPU memory smoke scripts:
scripts/trim_smoke_{metal,cuda}.jl.Out of scope
Outlier-only trimming (
policy=:largest) · usage-tracking / hysteresis · automatic periodic trimming · replacement/reserve buffers · a real (non-no-op) legacy implementation.