Skip to content

[Bug]: On Linux hosts without swap, the agents slice MemoryHigh throttles every agent for hours instead of OOM-killing the runaway #39195

Closed

Description

@matthias-scale

Area

apps/server

Steps to reproduce

  1. Run the T3 server as a systemd user service on a Linux host with 16 GiB RAM and no swap.
  2. Let several agents run builds, type checks and test runners at once, so the agents slice grows past MemoryHigh.
  3. Watch memory.pressure and memory.events (high) of the agents slice.

Expected behavior

A runaway agent or tool is OOM-killed inside its own cgroup, and the rest of the host keeps working.

Actual behavior

agentSliceMemoryLimits in apps/server/src/process/agentScope.ts sets MemoryHigh = max(RAM − 6 GiB, RAM / 2) and MemoryMax = max(RAM − 4 GiB, RAM × 0.75). On 16 GiB that is 10 GiB and 12 GiB.

Without swap, reclaim at MemoryHigh can only drop page cache. Anonymous memory has nowhere to go, so the kernel throttles every process in the slice and keeps throttling. MemoryMax is never reached, so nothing is killed and the stall does not end by itself.

On one 16 GiB host without swap (2026-11-10) the agents slice logged about 68,000 s (about 19 h) of full memory stall. All agents on the host were frozen in that time, with no OOM kill.

The limit is set again at every server start and on every RAM change (#30581), with --runtime drop-ins that outrank user drop-ins, so a host owner cannot turn it off in a lasting way.

Related:

Suggested fix

Workaround

After each server start, run systemctl --user set-property --runtime app-t3code-agents.slice MemoryHigh=infinity (re-applied by a timer), and put hard caps (MemoryMax, MemorySwapMax=0) on each agent scope through a systemd prefix drop-in, so one agent's OOM stays inside its own scope.

Impact

Major degradation or frequent failure

Version or commit

0.1.11-nightly.20261117.4825; code checked on main at cfd24da4a7

Environment

Linux x64, systemd user service, 16 GiB RAM, no swap, many concurrent agents

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions