Skip to content

feat(server): idle threads drop old tool output and reasoning to save space - #259

Merged
jayleaton merged 4 commits into
mainfrom
feat/thread-history-trim-20261117
Nov 17, 2026
Merged

jayleaton merged 4 commits into
mainfrom
feat/thread-history-trim-20261117

Conversation

@jayleaton

@jayleaton jayleaton commented Nov 17, 2026 •

Copy link
Copy Markdown
Owner

Problem

The database keeps every tool output and reasoning block for every thread forever. On the bigger machines it is tens of GB, and most of that is detail nobody reads again.

What this does

A thread nobody has used for 30 days keeps what you need to read, find and continue it, and drops the bulky detail.

  • Kept: user and assistant messages, plans, file changes, titles, runs and their status, parent/child links, linked PRs, settle and snooze state, checkpoints, handoffs and compaction items, search text.
  • Dropped: finished command output, tool calls, reasoning, file and web searches, and the tool/reasoning nodes behind them.
  • Also kept: tool output the timeline always shows inline (HTML renders, viewed images, sent messages, delegated tasks), which the upstream sync in chore: sync upstream t3code 9d28fc921b (v0.1.10) #258 marks with inline_work_output.
  • Shown: one notice in the thread where detail was removed.
  • Never trimmed: running, pinned or snoozed threads, threads with an open PR, side chats, and subagents whose root thread is still in use.
  • Idle means: the thread (or its root, for a subagent) was not created, opened, messaged or run within the cutoff.

Setting: Settings > Maintenance > Thread history (web and desktop), and the same setting on mobile. 0 turns it off; the default is 30 days, max 365. "Trim now" previews the count and size, then asks to confirm.

A trim pass runs once every 30 days. The server stores when the last pass finished (a row in the existing projection metadata table, so it survives restarts and updates; no migration). It checks five minutes after start and every six hours after that, and runs a pass when the last one was 30 or more days ago or none has run. So a machine that was off on the due date trims five minutes after its next start. "Trim now" always runs and resets the clock. Each pass logs a dry run first and trims only threads idle for the setting's days, at a low duty cycle. Space returns through the existing incremental vacuum loop, which now drains a backlog faster; no full VACUUM runs on a live database.

Measurements

Read-only, per machine. Sizes in MB.

Machine DB file Free pages already in file Trim saves at 30d at 60d at 90d
dev-box 26,479 14,503 2,405 (1,395 threads) 0 0
MacBook 10,226 7,643 1,128 (1,005 threads) 0 0
desktop-pc 987 251 17 0 0
Studio-01 4,893 0 0 (oldest idle thread is 18 days) 0 0
  • 60 and 90 days save nothing: threads that old predate the detailed storage.
  • On dev-box and MacBook the larger win is the existing "Reclaim database space" repair (14.5 GB and 7.6 GB of free pages). Those databases have auto_vacuum off, so trimmed space also needs that repair once.
  • desktop-pc barely benefits. Studio-01 starts saving in about two weeks.

Before and after

Seeded dev copy, 738 threads, cutoff lowered to 2 days so there was something to trim.

Before After
Threads trimmed 183 (42,395 items, 4.6 s)
DB file after vacuum 2,384 MB 1,921 MB
Full thread snapshot 110 ms, 64 MB 33 ms, 11 MB
Bounded thread open (what clients load) 5.6 ms 4.9 ms
Resume-history items 4,188 1,393

In the real web app: the trimmed thread reads with its notice, search finds it by message text, and a new turn on a trimmed thread completed through a context handoff (Replay provider).

Captures

Real web client, dev state. Full set: https://gist.github.com/jayleaton/1922d857d2a97a372d16cf5f422b265a

Before (main) After
Settings > Maintenance on main Thread history section

Preview and confirm, then the result:

Preview Trim now
Preview dialog Toast after confirming

The toast shot was taken as the dialog faded out, so the section behind it is mid-repaint. Nothing left to trim, and a trimmed thread with its notice expanded:

Nothing to trim Trimmed thread
Nothing to trim Trimmed thread notice

Schedule line ("Last trimmed · next") in the section:

Last trimmed and next pass

Tests

ThreadHistoryTrim.test.ts: a trimmed thread still reads, searches and builds resume history, and a second trim is a no-op; excluded threads are untouched; 0 disables it. The schedule test drives a TestClock through two server runs: a pass runs at start when one is due, not again within 30 days, again once due, right after a restart that missed its due date, and a requested trim resets the clock.

vp run check:changed passes with TMPDIR=/private/tmp.

Known limits

  • The size preview is an estimate. It counts each row and its index copies and reads about 80% of the real saving (411 MB estimated, 505 MB freed on a fresh seeded copy); the rest is page slack.
  • If a provider transcript is gone, the fallback handoff no longer includes command output from trimmed runs.
  • Find-in-thread can show stale results for a trimmed thread until its next event.
  • Mobile has the setting but no "Trim now" and no last/next line.
  • The first pass on each machine runs five minutes after its first start on this version, since none has run yet.
  • A due date that passes while the server is running is picked up within six hours, not to the minute.

Claude Fable 5.2, Claude Code

🤖 Generated with Claude Code

@github-actions github-actions Bot added vouch:trusted Author is trusted by repo permissions or the VOUCHED list. size:XL labels Nov 17, 2026
@github-actions

github-actions Bot commented Nov 17, 2026 •

Copy link
Copy Markdown

Thread transfer impact

✅ Thread transfer remains within every enforced ceiling.

Provider Metric Main baseline This PR Impact PR ceiling
Codex Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Codex Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Codex Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Codex Live turn WebSocket decoded 21.5 KiB 21.5 KiB 0 B (0.0%) 24.4 KiB ✅
Codex Live turn messages 1 1 0 (0.0%) 8 ✅
Claude Total thread wire 5.0 KiB 5.0 KiB 0 B (0.0%) 6.8 KiB ✅
Claude Thread snapshot wire 3.8 KiB 3.8 KiB 0 B (0.0%) 4.9 KiB ✅
Claude Live turn WebSocket wire 1.2 KiB 1.2 KiB 0 B (0.0%) 2.0 KiB ✅
Claude Live turn WebSocket decoded 21.9 KiB 21.9 KiB 0 B (0.0%) 24.4 KiB ✅
Claude Live turn messages 1 1 0 (0.0%) 8 ✅

Other scenarios

All 64 scenario measurements match main.

All 64 scenario measurements
Provider Scenario Metric Main baseline This PR Impact PR ceiling
Codex short turn resume WebSocket decoded 10.5 KiB 10.5 KiB 0 B (0.0%) 12.7 KiB ✅
Codex short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn running, cold open HTTP decoded 281.7 KiB 281.7 KiB 0 B (0.0%) 302.7 KiB ✅
Codex long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Codex long turn running, history page HTTP decoded 92.4 KiB 92.4 KiB 0 B (0.0%) 107.4 KiB ✅
Codex long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Codex long turn finished, cold open HTTP decoded 381.5 KiB 381.5 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, socket cold open WebSocket decoded 381.6 KiB 381.6 KiB 0 B (0.0%) 410.2 KiB ✅
Codex long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Codex long turn finished, work page HTTP decoded 186.2 KiB 186.2 KiB 0 B (0.0%) 200.2 KiB ✅
Codex long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Codex idle thread resume WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell snapshot HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Codex fleet shell subscribe WebSocket decoded 65 B 65 B 0 B (0.0%) 200 B ✅
Codex fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Codex fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Codex fleet shell resume check rows 0 0 0 0 ✅
Codex stale shell reload HTTP decoded 42.4 KiB 42.4 KiB 0 B (0.0%) 53.7 KiB ✅
Codex stale shell reload rows 27 27 0 (0.0%) 27 ✅
Codex stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Codex stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Codex stale todo list cold open HTTP decoded 72.4 KiB 72.4 KiB 0 B (0.0%) 83.0 KiB ✅
Codex stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Codex stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Codex stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Codex stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅
Claude short turn resume WebSocket decoded 10.6 KiB 10.6 KiB 0 B (0.0%) 12.7 KiB ✅
Claude short turn resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn running, cold open HTTP decoded 283.3 KiB 283.3 KiB 0 B (0.0%) 302.7 KiB ✅
Claude long turn running, cold open rows 75 75 0 (0.0%) 75 ✅
Claude long turn running, history page HTTP decoded 93.4 KiB 93.4 KiB 0 B (0.0%) 107.4 KiB ✅
Claude long turn running, history page rows 75 75 0 (0.0%) 75 ✅
Claude long turn finished, cold open HTTP decoded 384.2 KiB 384.2 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, socket cold open WebSocket decoded 384.3 KiB 384.3 KiB 0 B (0.0%) 410.2 KiB ✅
Claude long turn finished, socket cold open WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude long turn finished, socket cold open rows 151 151 0 (0.0%) 151 ✅
Claude long turn finished, work page HTTP decoded 188.1 KiB 188.1 KiB 0 B (0.0%) 200.2 KiB ✅
Claude long turn finished, work page rows 150 150 0 (0.0%) 150 ✅
Claude idle thread resume WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude idle thread resume WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell snapshot HTTP decoded 43.0 KiB 43.0 KiB 0 B (0.0%) 53.7 KiB ✅
Claude fleet shell snapshot rows 27 27 0 (0.0%) 27 ✅
Claude fleet shell subscribe WebSocket decoded 66 B 66 B 0 B (0.0%) 200 B ✅
Claude fleet shell subscribe WebSocket messages 1 1 0 (0.0%) 1 ✅
Claude fleet shell resume check HTTP decoded 18 B 18 B 0 B (0.0%) 100 B ✅
Claude fleet shell resume check rows 0 0 0 0 ✅
Claude stale shell reload HTTP decoded 43.0 KiB 43.0 KiB 0 B (0.0%) 53.7 KiB ✅
Claude stale shell reload rows 27 27 0 (0.0%) 27 ✅
Claude stale shell resume WebSocket decoded 140 B 140 B 0 B (0.0%) 400 B ✅
Claude stale shell resume WebSocket messages 2 2 0 (0.0%) 2 ✅
Claude stale todo list cold open HTTP decoded 73.4 KiB 73.4 KiB 0 B (0.0%) 83.0 KiB ✅
Claude stale todo list cold open rows 75 75 0 (0.0%) 75 ✅
Claude stale todo list cold open nodes 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open subagents 2 2 0 (0.0%) 4 ✅
Claude stale todo list cold open plans 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open providerTurns 1 1 0 (0.0%) 2 ✅
Claude stale todo list cold open checkpoints 1 1 0 (0.0%) 2 ✅

Baseline: 59469d4 · PR result: 75a06b5 · Source CI: success

Scenario and decoded snapshot size

10 historical turns, 5 command tools per turn, 878.9 KiB retained MCP result per historical turn, and a 1.05 MiB retained result in the measured turn.

  • Codex decoded thread snapshot: 108.5 KiB
  • Claude decoded thread snapshot: 108.8 KiB

Updated in place by a trusted workflow. PR artifacts are strictly validated and never executed.

@jayleaton
jayleaton marked this pull request as ready for review November 17, 2026 04:02
@github-actions github-actions Bot added size:XXL and removed size:XL labels Nov 17, 2026
jayleaton and others added 4 commits November 17, 2026 13:09
… space

Threads nobody has used for 30 days (setting, 0 turns it off) keep their
messages, plans, file changes, runs, checkpoints and handoffs, and lose
finished tool output and reasoning. A notice marks where detail was removed.
Running, pinned, snoozed, open-PR and side-chat threads are skipped. The
sweep logs a dry run at start, runs an hour later, then every six hours.
Settings > Maintenance has the setting, a preview and "Trim now".

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
…ayload

The estimate read about half the real saving because it ignored the ids
each row repeats in its table, indexes and position row. It now reads
about 80% (411 MB estimated, 505 MB freed on a seeded copy).

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
… start

The server stores when the last trim pass finished. A pass runs when that
is 30 or more days ago: while running, within six hours of coming due;
after a missed due date, five minutes after the next start. "Trim now"
always runs and resets the clock. Settings > Maintenance shows the last
and next pass.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
The upstream sync marks HTML renders, viewed images, sent messages and
delegated tasks with inline_work_output. Those are part of what a reader
sees, so a trim pass leaves them and their parents in place.

Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
@jayleaton
jayleaton force-pushed the feat/thread-history-trim-20261117 branch from 2221dcc to 75a06b5 Compare November 17, 2026 05:12
@jayleaton
jayleaton merged commit b34c41b into main Nov 17, 2026
48 of 51 checks passed
@jayleaton

Copy link
Copy Markdown
Owner Author

Post-merge review, by Randy, at d6db6c5d5. This PR merged at 75a06b5, which adds inline-output retention (inline_work_output), the 30-day pass clock and full-row preview sizes. These points are still open on main, and a follow-up PR will address them:

  1. Blocking: apps/server/src/orchestration-v2/ThreadHistoryTrim.ts (~308). Activity during a pass doesn't stop deletion. Eligibility is checked once per thread, and later pages only check item status. In a reproduction, the thread was visited right before deletion, and three items were still removed. Fix: recheck idle eligibility and the current setting inside every delete transaction, and stop when the thread becomes active or trimming is off.
  2. Blocking: ~275. Deletes bypass revisions and cache invalidation. The notice event is written before the deletes, and no completion event names the removed rows. The same snapshot sequence (14) returned 8 items, then 5. Connected clients and the timeline and search caches keep stale state. Fix: write a durable trim revision after deletion, invalidate the affected caches, and test reconnect and paging across a pass.
  3. Nit: ~185. The preview counts retained ancestors. The preview showed 4 items / 12,974 bytes, but the trim deleted 3 items / 7,446 bytes. Fix: use the same selection for the preview and the delete.
  4. Needs tests: inline outputs survive a trim. Cover send_to_user, HTML renders and embedded apps in the web and mobile timelines, and in exports and transfers, now that inline_work_output is in.
  5. Product question, for JJ: default-on for existing installs. Missing settings decode to 30 days, and the first pass runs 5 minutes after start. There's no acknowledgement or archive, setting 0 can't restore anything, and pruned events mean replay can't undo it.

The 41 targeted tests, the changed-file lint and the typechecks passed at the reviewed head.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL vouch:trusted Author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant