Skip to content

Idle compaction never runs while a delegated task or background command is pending, so orchestrator threads always resume with a cold cache #39715

Closed

Description

@StantonMatt

What happened

"Compact idle threads: Before the cache expires" is turned on for a Claude instance (subscription auth, 1-hour cache). It works for threads that are truly idle. But it almost never runs in orchestrator threads, which are the threads it matters most for. An orchestrator usually ends its turn while a delegate_task child, a background command or a PR watch is still running. The parent's prompt cache then expires while T3 waits. When the child reports back, the wake turn re-reads the parent's whole context uncached. When I return first, the composer's compact-before-send runs /compact against an already-cold cache.

Measurements (past 7 days, one desktop, 0.1.11-nightly.20261120.4979)

From statev2.sqlite: I looked at top-level Claude threads whose last turn ended above 200K tokens and checked what happened next, in relation to promptCacheExpiresAt on the latest provider turn.

Outcome Count Context tokens
Server idle compaction (beforeCacheExpires) ran in time 42 15.2M summarized
/compact ran after the cache expired (onReturn, from the composer) 8 4.3M re-read cold
Thread woke after the cache expired and continued without compacting 30 16.6M re-read cold

What woke the threads in those 30 cold continuations:

  • 24 were a delegate_task completion delivered to the parent.
  • 5 were a background command finishing.
  • 1 was a pull request update.

They were 214K–915K tokens each, all in four orchestrator threads.

For the 8 late compactions: in 5, a delegated child of that thread was still running when the compaction window opened (expiry − 2 min). I couldn't explain the other 3 from the database; they may be server restarts (#27915).

So of 79 cases where a large thread's cache was about to expire, 37 ended with an uncached re-read of about 20.7M tokens. At least 29 of the 37 are explained by this gate.

One concrete case: the parent's last turn ended at 11:36:27Z with 322,888 tokens, and its cache expired at 12:36:27Z. A delegate_task child (another provider instance) had been running since 10:05:38Z and was still running. No server /compact was sent. The user returned at 12:53Z and got the composer's Compact/Keep toggle with a cold cache.

Diagnosis

idleCompactionWindow() in apps/server/src/orchestration-v2/IdleCompaction.ts returns null whenever derivePendingBackgroundWork(...) is non-empty:

if (runtimeRequests.some(...) || contextTransfers.some(...) || derivePendingBackgroundWork({
  latestRun, runs, providerThreads, turnItems, activeProviderThreadId: thread.activeProviderThreadId
}).length > 0) return null;

derivePendingBackgroundWork counts every active turn item of type subagent, command_execution or dynamic_tool, plus the provider thread's pendingBackgroundTasks. A delegate_task child is recorded in the parent as a subagent turn item that stays running until the child's result is delivered. That can be hours later, so the window is null for the whole cache lifetime. (PR watches aren't passed in here, so they don't block this check by themselves. I haven't checked why the one PR-update wake didn't compact.)

compactWhenDue evaluates the window once, when the parent's run completes, and returns as soon as it is null. It isn't re-armed when the background work later finishes without a new parent run. In practice the child's report starts a new parent run anyway, but by then the cache is cold.

The decider has a second guard for idleCompaction === "beforeCacheExpires" that drops the command if any run is queued or active, or if a delegated completion is open but undelivered. That one looks right: it only covers work that is about to start a run.

Why the gate looks too broad

A delegate_task child runs in a separate thread and provider session, often on another provider. It never touches the parent's context until its result arrives as a new user message. Compacting the parent while the child runs loses nothing that the parent would otherwise use. Not compacting means the eventual delivery turn always pays a full uncached read once the child takes longer than the cache TTL. That is common for implementation workers.

Suggested fix

In idleCompactionWindow(), stop treating T3-owned delegated children (subagent items with a childThreadId, and other cross-instance children) as blocking. They can't be affected by compacting the parent.

For in-process work, it depends on whether the Claude session can /compact safely while that work runs:

  • native run_in_background commands
  • native Claude subagents
  • monitors

If it can, drop the background-work condition entirely and keep only the decider's queued/active-run guard. If it can't, keep blocking on those items, and re-check the window when the last blocking item finishes (rather than only on run.updated completed). A thread whose background command ends at minute 40 could then still compact before minute 60.

Related: #27915 (timers lost on server restart), #30155 (delegated completions waking a parent whose cache has expired).

Version

T3 Code desktop 0.1.11-nightly.20261120.4979 (Linux AppImage), Claude provider with subscription auth (1h prompt cache), global idleCompaction: "beforeCacheExpires".

Activity

  1. StantonMatt commented on Nov 20, 2026

    @StantonMatt
    Author

    Some context I missed when filing:

    Related open work in the same code that doesn't change this gate: #27926 (re-arm after restart, which may explain the 3 unexplained late compactions), #27742 (it reuses the same eligibility, including the background-work gate) and #37519.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions