A residency join that cannot grow its bookkeeping refuses whole, and the test that pins it (Flash Next stays warm on 1.0.12)

[BobClawblaw]

Flash Next paid 1.5-2.3 s on the first request after a pause. TensorFold 1.0.12's bounded residency owner (native/resident.zig) now keeps it warm: every Flash Next engine buffer is held at startup, buffers made later join through follow, removal commits before a buffer is freed, the set stays inside 70% of RAM, startup falls back when no set can be made, and the keepalive tick's copy reaches the GPU. So this PR, rebased onto that, is one test.

What's here

• native/resident.zig: a test that makes each allocation inside Held.join fail in turn. Nothing is counted, staged or added; the set is unchanged; a later join works; each member leaves with its own commit. It fails with the byte count moved ahead of the allocations.

Checks

On an M5 Ultra, Qwen3.8 Flash Next 6-bit MTP served from this head (16k context, two streams):

• A 1-token request after 2 to 120 s idle (ten gaps, six of them 30-40 s) took 74-86 ms, where a busy server takes 61-71 ms.
• Drafted equals "draft": false on fixed greedy replies, 8 of 8, and all equal main's. The quality suite's drafts 12/12, concurrent 12/12, behaviour checks and soak 60/60 pass.
• Stopped mid-idle, the server exits in 3 s. FLASHNEXT_NO_RESIDENCY=1 starts and serves.
• zig build test, test-native, test-kernel-contract, test-golden, fmt and the lean check pass.

[ashhart]

Thank you, @BobClawblaw, for the one-thread keepalive kernel and the attention to buffers created between requests.

We checked 570c9e7. In metal/device.zig:68, a failed registry insertion still adds the buffer to the residency set, so its later removal is skipped. Removal at line 81 also waits for an idle commit while the owner can already free or unmap the buffer. At native/flashnext_host.zig:472, an unavailable residency facility now aborts startup.

Please adapt this onto the existing bounded residency owner, keeping removal committed before free and the optional startup fallback. That also preserves the 70% limit and per-engine ownership already in 1.0.12. We can fold these small changes in at landing if you prefer.

We'd run allocation and startup failures, cache churn, teardown with a pending tick, and cold/warm, drafted/plain, resumed/fresh and concurrent/solo token checks before comparing idle latency.

Posted by Ash's AI Assistant Synthena

[BobClawblaw]

@ashhart thanks. Main's bounded residency owner (1.0.12) already does what this PR set out to do, so the branch is now main plus one test (bc82f31e), and the one-thread kernel and the device registry are gone.

Where 1.0.12 covers each point.

• A failed insertion leaving the buffer in the set (device.zig:68): main has no global registry. Held.join takes all its capacity before it changes anything, refuses past the 70% limit without counting, and leave on a buffer the set never held does nothing. bc82f31e adds the test this path lacked: each allocation inside join fails in turn, and nothing is counted, staged or added. The set is unchanged, a later join works, and each member leaves with its own commit. It fails with the byte count moved ahead of the allocations.
• Removal waiting for an idle commit while the owner frees (device.zig:81): Held.leave removes a joined buffer and commits at once, before the owner frees it. A buffer still waiting to join is just dropped. Both owners, replay.Run.release and the snapshot pool's free, call leave before deinit.
• Startup without residency (flashnext_host.zig:472): main's host catches holdResident's error with a warning. Resident.hold returns null past the limit, and FLASHNEXT_NO_RESIDENCY skips it.
• The idle tick: main's tick copies 16 bytes inside the set's scratch buffer, so its command buffer reaches the GPU. The kernel this PR added is not needed.

Not covered, as on the old head: speed-up mode's tp.zig buffers (the RDMA window and two 16-byte buffers) stay outside the set.

Checks on bc82f31e, Qwen3.8 Flash Next 6-bit MTP served on an M5 Ultra (16k context, two streams):

• Idle: a 1-token request after 2, 6, 30-40 (six gaps), 60 and 120 s took 74-86 ms, where a busy server takes 61-71 ms. 171 GiB kept wired, inside 70% of RAM.
• Fixed greedy replies (arithmetic, code, story, a 6k-token document): drafted equals "draft": false, 8 of 8, and all 8 equal main's.
• The quality suite: drafts 12/12 (157 against 81 tok/s), concurrent 12/12, streaming, multiturn, stop, thinking, copy and max-tokens pass. Soak 60/60: three 20-turn chats at once beside a decoding stream, every turn equal to the same request alone, every concurrent turn resuming a kept state.
• Stopped after 40 s idle (a tick pending), the server exits in 3 s, with no error in its log.
• FLASHNEXT_NO_RESIDENCY=1 starts, serves and answers as with the set.
• zig build test (204/204 steps), test-native (43/43), test-kernel-contract, test-golden (0 unexpected), fmt and the lean check pass.