Repository navigation
Description
What happened
A Claude agent ran a Python test that loaded a model on the CPU, and the machine ran out of memory. The kernel killed only the Python process:
kernel: oom-kill:constraint=CONSTRAINT_NONE,...,global_oom,task_memcg=/user.slice/user-1000.slice/user@1000.service/app.slice/app-t3code.slice/app-t3code-agents.slice/app-t3code-agents-7a9722fa0109.slice/t3code-claude-8fdb8ce3.scope,task=python,pid=443587
kernel: Out of memory: Killed process 443587 (python) total-vm:28182680kB, anon-rss:4431432kB, ... oom_score_adj:800
systemd[1815]: t3code-claude-8fdb8ce3.scope: The kernel OOM killer killed some processes in this unit.
systemd[1815]: t3code-claude-8fdb8ce3.scope: Failed with result 'oom-kill'.
Because the scope has OOMPolicy=stop, systemd then stopped the whole scope. That took down the claude process and the two background subagents it was running. The turn failed with "Killed: out of memory", and the user had to type "continue". The agent later explained: "the shared-prefix test ran the 0.8B model in full precision on lenovolinux's CPU, and that took the session and both helpers down with it."
Without the stop, the Bash tool would have returned exit 137 for that one command, and the agent could have run the test somewhere else.
Background
- fix(server): agents run in their own systemd scopes so an OOM kill spares the server #17662 chose
OOMPolicy=stopfor agent scopes on purpose, so systemd recordsResult=oom-killand the failed turn can say "Killed: out of memory". - fix(server): keep the shared Codex app-server scope alive after a tool OOM kill #18329 switched the shared Codex app-server scope to
continue, and fix(opencode): a tool killed for memory no longer stops every OpenCode thread #19367 did the same for OpenCode. fix(opencode): a tool killed for memory no longer stops every OpenCode thread #19367 says that changing per-thread scopes "is outside this focused fix", and that the default change needs human approval. - Luzivog reported this same Claude case in a comment on [Bug]: One OOM-killed tool command stops the shared OpenCode scope and every OpenCode thread shows "Run interrupted by user" #19164: a 2 GB dev server's kill took Claude and two subagents with it. That issue was closed after only the OpenCode part was fixed.
The trade-off
Switching thread scopes to continue (apps/server/src/process/agentScope.ts) is not a one-line change:
classifyScopetreatsoom_kill > 0as "stopping". After a tool kill the scope survives, so every later failed turn would poll for about 2 seconds inoomKilled().stop()leaves scopes with a non-zero counter for systemd to stop. Undercontinue, nothing stops them, so tool processes would outlive the agent.- The "Killed: out of memory" label would partly go away:
- When
claudeitself is the victim:memory.eventscan still show the kill, but only if the cgroup still exists when it is read, which races onceclaudewas the last process. - When systemd-oomd does the kill: the label is lost entirely, because oomd does not move the kernel counter.
- When
Two alternatives don't fit:
memory.oom.groupkills the whole group, which is the opposite of what's wanted.- Per-tool scopes aren't possible, because Claude Code starts its own Bash children inside its cgroup.
The fix I'd propose:
- Use
continuefor thread scopes. - Stop treating a non-zero counter as "systemd is stopping this" in
classifyScopeandstop(). - Label a failed turn from a counter increase together with the agent's own SIGKILL exit.
- Accept that the label is lost for oomd kills and in the race above.
That's roughly 50 to 100 lines with tests. I can send a PR if you want that trade.
Version or commit
Nightly 0.1.10-nightly.20261116.4800; agentScope.ts is unchanged on main at 1d20ad0.
Environment
Ubuntu 26.04, systemd 259, kernel 7.0, 16 GB RAM, Claude Code 2.1.326, Claude Opus 5.5.
Activity
fsioni commentedon Nov 17, 2026
+1, hitting this on a second machine, with one difference worth noting: here the killer is systemd-oomd, not the kernel OOM killer, so
OOMPolicy=continuealone would not help.Setup: Ubuntu, 16 cores / 26 GB RAM, nightly 0.1.8 build, Claude agents compiling and testing a Rust workspace in several worktrees at once.
In 45 minutes, oomd killed nine whole Claude agent scopes. Every one of those turns failed with "Killed: out of memory":
user@.servicehas Ubuntu's defaultManagedOOMMemoryPressure=killat a 50% limit. The agents slice hadMemoryHigh=21.9GandMemoryMax=24.1G.What this adds to the analysis above:
OOMPolicy. Switching thread scopes tocontinuefixes the kernel case but not this one. Each scope still holdsclaudetogether with every tool it spawned.MemoryHighthrottle on the agents slice may make this worse. Throttling forces reclaim, the pressure that oomd measures rises, and oomd can then kill a scope beforeMemoryMaxis ever reached.Result=oom-killfor the oomd kills. Point 4 above may be too pessimistic, at least on this systemd version.A fix for this case probably needs the agent's tools in a child cgroup that oomd can pick instead of the agent. Another option is
ManagedOOMPreference=avoidon the agent process, though that only works for cgroups the user owns.Investigated with Claude Code (Opus 5.5).
caniko commentedon Nov 17, 2026
Same on a NixOS workstation (kernel OOM killer, not oomd). On 2026-11-14 single tool kills (rust-analyzer, nix, a linker) stopped nine Claude sessions mid-turn.
We've run per-thread scopes with
OOMPolicy=continuesince then in a fork: caniko@3d53030 (12 lines added, 24 removed inagentScope.tsand its test). Besides the policy, it drops the early return instop()that left scopes with a non-zero OOM counter for systemd, so a scope is reaped when its agent exits even after a tool was killed. That covers your second point. It doesn't touchclassifyScope, so the ~2 soomKilled()poll after a tool kill and the "Killed: out of memory" label are still open. Withcontinue, the killed tool fails its own command and the agent keeps its turn.(Written with Claude Opus 5.5 in Claude Code.)