Repository navigation
Conversation
frodosik
force-pushed
the
fix/unloaded-restore-and-sse-client-gone
branch
from
November 16, 2026 18:57
6a88da0 to
7974e44
Compare
An engine unloaded by --lazy / idle_unload_s reports no context and no KV type in /metrics. Engine() read that as a fingerprint, so a request against an unloaded engine evicted every session file. It now refuses an incomplete /metrics; a restore against an unloaded engine uses the last fingerprint learned, or the newest file on disk, wakes the engine with a 1-token request, checks that it is the engine the file was written for, and retries the restore once. A refused wakeup, or a woken engine with another fingerprint, is reported as what it is, not as the initial 503. A cold engine with no files passes through untouched. Store.AnyFingerprint() breaks a tie between two files of the same mtime by name, so the answer does not depend on map order. A client that closes the connection right after the stream's terminal event got a complete answer, not a cancellation. Every chunk written is scanned for the terminal event and a sticky flag remembers it, so a Responses completion followed by kilobytes counts; [DONE] is anchored to its line, and the rolling tail is seeded with a newline once at stream start, so trimming it cannot invent a line start for a marker quoted inside streamed content. A cancel while the stream runs is still not trusted as live state. A file that restores less than half the tokens its name claims is forgotten, so the conversation is saved afresh instead of being skipped as already on disk. Checked against Strata v0.1.45. New tests in internal/strata, internal/store and internal/proxy, against stratatest, which now simulates an unloaded engine.
frodosik
force-pushed
the
fix/unloaded-restore-and-sse-client-gone
branch
from
November 16, 2026 19:06
7974e44 to
130f8d0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Three bug fixes for production failures seen with
--lazy/idle_unload_sand SSE agent harnesses.1. An unloaded engine made every session file look foreign (
internal/strata/client.go,internal/store/store.go,internal/proxy/cache.go)./metricsof an unloaded engine reports nocontextand nokv, so the fingerprint hashcontained the literal
<nil>;learn()recorded that as a new fingerprint andKeepFingerprintdeleted every.sessionfile on disk — the file that should restore theconversation the engine just unloaded.
Engine()now refuses an incomplete/metrics, anda switch against an unloaded engine restores with the fingerprint it last learned, or the
newest file's, so
Lookupcan see the file without waking Strata:The restore itself wakes the engine. Strata answers the refused slot call with
503 "not loaded", and a 1-token request loads the model:Store.AnyFingerprint()is the fingerprint of the most recently used file on disk; twofiles with the same mtime break the tie by name, so the answer does not depend on map
order. A refused wakeup, or a woken engine of another fingerprint, is logged and counted as
what it is, not as the initial 503.
2. A client that closes the socket after the stream finished counted as a cancellation
(
internal/proxy/proxy.go). SSE harnesses (OpenAI SDKs, agent loops) read to[DONE]andclose; the proxy saw the request context cancelled, set
client_gone, droppedlive, andthe next turn was read fresh — the answer it had just served was never trusted, and the
idle save after a completed turn never ran. Every chunk written is scanned for a terminal
event, and the answer is remembered in a sticky flag:
A marker is searched in the whole chunk, not in a fixed window at the end, because
response.completedis followed by the entireresponseobject — kilobytes that push themarker out of any tail.
[DONE]is matched as\ndata: [DONE], and the newline isseeded once at stream start: trimming the tail to 64 bytes can leave it starting right at
a quoted
data: [DONE]inside JSON content, and the next chunk must not invent a linestart for it. Inside JSON strings newlines are escaped, so a model quoting
data: [DONE]in streamed content is not the end of the stream.cache.finishkeeps settingc.live = nilon a genuine mid-stream cancel, so a cut-offgeneration is never trusted as live state.
3. A truncated
.sessionfile trapped the conversation (internal/proxy/cache.go). Afile whose name claims more tokens than the engine restores from it — a save cut short, an
interrupted rename — is
Coveredfor every later turn, so every save is skipped ason_diskwithinregrow-tokensand the conversation never reaches disk again. A restorethat returns fewer than half the tokens in the name forgets the file:
stratatestgained what the tests need: an unloaded engine (/metricswithout contextand kv, slot calls refused 503), a request that loads it back, a refused generation
request (
GenStatus),SlotMessage,Finish,Content,TailDelay, terminal events inall three dialects with the Responses
response.completedcarrying a multi-KB payload,and a restore that reports its own token count instead of a fixed 20000.
docs/design.mdgets the cases with their tests;
CHANGELOG.mdgets a### Fixedsection. Request bodiesare not touched.
Evidence
Production symptom, the turn after the engine unloaded (file names and timings shortened):
After the fix, the same turn restores from disk (from
TestColdEngineWakesAndRestores):go vet ./...clean,golangci-lint(v2.12.2, repo config) 0 issues,go test -race -count=1 ./...all green:TestClientGoneAfterCompletionkeeps the cancellations honest: a cancel while the streamruns, and a cancel at an answer that quotes
data: [DONE]inside"content", are bothrecorded
client_gone=trueand the next turn is read fresh.Merge Danger
Door: two-way. One commit, no flag, no wire-format, storage-format, metric-name, or
config change; reverting it restores the previous behaviour.
Blast Radius: cache decisions only — which fingerprint a restore matches against,
when the engine is woken (one extra 1-token request per restore against an unloaded
engine), when
client_goneis cleared, and when a file is forgotten. Prompt and answertext, request bodies and the log fields are unchanged, except
client_gone's value.Checked against Strata v0.1.45.