ServerPilot

Agents take their own GPUs. You watch the whole fleet.
MCP for agents · Native desktop app · Open source

中文 · What it does · Quick start · Agent usage · Boundaries · Docs

_{Illustration only; not a real resource state or app screen.}

Your agents write code and run experiments on their own. Are you still handing them GPUs one card at a time?

To an agent, ServerPilot is an MCP server: look at cards, take some, give them back, and register or update a host. To you, it is a macOS app showing what is free, what is busy, whose job is where, and what is failing across every server you registered. There is no browser UI.

One local user, several servers, several agents. Resource state, requests, and human correction all read the same committed snapshot from one local control plane.

What it gives you	How
One source of truth	The GUI, the CLI, and MCP all read the daemon's committed snapshot instead of each deriving state.
A three-step loop for agents	gpu_status → gpu_apply → gpu_release covers routine coordination.
A view for humans	Servers, tasks, ownership, and failures, with bounded manual correction when something is wrong.
Per-card idle holding	Optional keepalive yields only the card being requested and restores the rest.

✨ What it does

🛰️ One picture of every server

Server state refreshes on the collection interval you set. Free cards, busy cards, which task owns what, and collector failures are all visible in the app. Servers can sit in a first-class group that carries a shared workspace plus environment and data/weight notes; a member inherits that workspace or overrides it. Environment notes are descriptive only. Allocatable capacity is shown group → server → SKU, not as a menu of individual free cards.

🧩 Agents allocate for themselves

The MCP surface is exactly five tools:

• 🔍 gpu_status — grouped allocatable capacity, what is busy and to whom, and telemetry for cards you already hold
• 🔑 gpu_apply — take GPUs (server_group_id?, server_id?, gpu_count=1, task?)
• ♻️ gpu_release — give them back
• ➕ gpu_add_server — register a host
• ✏️ gpu_update_server — update safe host metadata

A successful request returns the SSH connection, the remote working directory, a CUDA selector, and a lease_id, so an agent never has to guess a server, a directory, or a GPU index. Deleting a server stays in the app and REST; it is not an MCP tool, and it refuses while that server holds active leases.

🛠️ You step in only when something is wrong

While agents work normally you only watch. When ownership looks wrong, a connection fails, or a hold is left behind, you confirm and correct it: settle a stale lease, or move a task to different GPUs while keeping the card count.

🐶 Optional: hold idle GPUs so nobody else takes them

Per-card keepalive can keep confirmed-idle GPUs on standby. When an agent requests one, only that card is yielded, and it returns to standby after release. Other cards on the same machine are untouched.

Use it only on servers you administer. It never touches an unknown process or someone else's running job.

Keepalive workers run detached from the control plane. If the control plane stops, those GPUs stay occupied until it comes back and reconciles, or until someone stops them. There is no automatic release on shutdown.

When the control plane is unreachable, these commands put things back. None of them needs the daemon to be running. They act on the macOS control plane's data directory, so they do not apply to a Windows install:

serverpilot keepalive inspect --endpoint <server-id>   # what is still holding cards
serverpilot keepalive stop --endpoint <server-id>      # stop it and free them
serverpilot daemon reclaim                             # take port 8787 back

daemon reclaim only resolves the case where a ServerPilot service answers on 8787 without launchd owning it; when the daemon is properly owned it does nothing. The port-ownership error names the holding process and its command line.

🚀 Quick start

1. 🧰 Start the local control plane

On macOS the CLI is the backend. uv tool install installs the control plane the daemon will run; the desktop app is only the GUI. Opening the app does not start or replace that process.

macOS, from source, with Python 3.12+ and uv:

git clone https://github.com/JinPLu/ServerPilot.git
cd ServerPilot
uv tool install --force .
serverpilot daemon install --source-root "$PWD"
serverpilot daemon status

daemon install registers a user LaunchAgent that starts the uv tool install, and is macOS-only. After a later upgrade, confirm the process on http://127.0.0.1:8787/health/live actually reports the new version — see the upgrade checklist (Chinese).

Windows: the desktop app is macOS-only, but the CLI and MCP entry points work on Windows. From source, use serverpilot serve --db <path> --inventory <path> and keep that process running; there is no supervised install on Windows yet.

Either way it listens on http://127.0.0.1:8787.

2. 🖥️ Register your GPU servers

Nothing is installed on the GPU servers. ServerPilot reads them with one fixed, read-only SSH probe, so a server is ready as soon as you can reach it:

ssh user@host true          # must succeed without a prompt

The probe runs in a non-interactive SSH session, which is where PATH differs from the one you get logging in by hand. On a GPU host it expects nvidia-smi to be on that PATH; a host without it is registered as a CPU-only machine rather than rejected.

ssh user@host nvidia-smi -L  # on a GPU host, must list the cards

Then add the SSH connection and an absolute remote working directory in the app. A GPU becomes allocatable only after a fresh collection succeeds. Bare metal uses the built-in linux observation profile, a fixed read-only SSH probe; nothing needs to be installed on the remote host. The full contract is in observation and keepalive (Chinese).

Do not register a shared cluster (Slurm, LSF, PBS) as bare metal. A local plugin takes over observation so only your own jobs are registered, and requests go through that plugin's apply / release. The bundled slurm-immediate plugin is the reference; there is no separate scheduler submission surface. See observation and keepalive for the contract.

3. 🤖 Connect an agent

serverpilot mcp install --client codex     # or claude, cursor
serverpilot mcp policy --install --client codex

serverpilot mcp install only writes the launch command: Codex and Claude Code get their mcp add, and Cursor is merged into ~/.cursor/mcp.json without disturbing servers already there. It does not refresh a client's cached tool list — reconnect the server (Cursor: Disable → Enable, or reload the window) so it runs tools/list again.

To paste it yourself, serverpilot mcp config --client all prints the registration without writing anything. The standard block is:

{
  "mcpServers": {
    "serverpilot": {
      "command": "serverpilot-mcp",
      "env": { "SERVERPILOT_URL": "http://127.0.0.1:8787" }
    }
  }
}

The CLI and MCP entry points work the same way on Windows; the desktop app is macOS-only. Full per-client notes are in the Agent / MCP guide.

🤖 How an agent uses it

gpu_status → gpu_apply(server_group_id=<group>, gpu_count=<launch config>, task="my task") → use what it returns → gpu_release(lease_id)

• Routine apply is gpu_apply(server_group_id?, server_id?, gpu_count=1, task?). Agents never pass GPU IDs; gpu_apply picks the cards, and one lease always lands on a single server. gpu_count is exact job parallelism from the launch script or config. The safe default is 1. Never infer it from free capacity.
• On a grouped host — bare-metal direct or plugin delegated — pass server_group_id; the broker then best-fits one host inside that group. server_id is only for ungrouped hosts. A plugin-adapted cluster appears as an ordinary group, with allocation, limits, and largest_allocatable_block (one apply's max cards, not remaining pool size; null means unknown — do not invent a number).
• Inventory uses gpu_add_server and gpu_update_server. There is no MCP delete; remove a server in the app or REST, which refuses while it holds active leases.
• Assess a group's workspace, environment notes, and data/weight notes before claiming. Endpoints inherit the group workspace or override it. Environment notes are descriptive only — they are not executed or injected.
• Allocatable capacity is grouped group → server → SKU (name, vram_mib, total_count, available_count), not a per-free-card menu, and carries no telemetry. Load observable on a free card comes from ServerPilot's own keepalive hold, which is stopped before the card is handed over, so it is not evidence that the card is busy. Telemetry follows the lease: gpu_status(lease_id=…) returns rolling ten-minute averages for leased_gpus plus a lease summary (min_memory_free_mib, slowest_gpu) for judging whether your own job is using the cards well.
• SSH in, cd to the returned workspace.path, then apply the CUDA selector. That path is a working directory, not a code repository.
• Release immediately if CUDA fails to initialise or the workload does not start.
• no_capacity means nothing was allocated and nothing was queued. It comes back as data, not as an error, and it is not worth retrying in the same turn.

🖥️ The desktop app

The desktop app is macOS-only; the CLI and MCP entry points work on Windows too. The Settings page shows this installation's MCP entry as an absolute path and the pasteable mcpServers JSON. Copy either into Codex, Claude Code, or Cursor. If the executable is missing, the same panel says so and repeats the install hint instead of inventing a path.

macOS

Install the CLI first — that is the backend. Then open the desktop app to watch state, ownership, and failures. The app does not run its own control plane.

open "./ServerPilot.app"

There is no browser UI. Manual reassignment updates the lease and the CUDA selector; it does not migrate a running process, so the agent has to restart its workload against the new selector.

🛡️ Boundaries and security

• ServerPilot manages the lifecycle of its own occupancy processes and of plugin-side allocations. It does not start, stop, migrate, or preempt your workloads. Keepalive starts and stops a per-GPU CUDA process on the remote host holding about 80% of that card's VRAM, and gpu_apply stops it before handing the card over. A plugin that declares apply / release performs the matching cluster allocation on request and release.
• Server state comes from fixed collection: the built-in linux SSH probe (a fixed read-only query, nothing installed on the remote), or a local plugin's observe. The plugin calling contract is four fixed verbs — info, observe, apply, release. No arbitrary remote command is accepted, and no password or private key is provided. See OBSERVATION_zh.md.
• Stale collection, connection errors, unknown processes, and resource conflicts all refuse allocation locally. That holds for what collection reports: the SSH user is trusted, and for a delegated cluster the plugin executable is trusted. A replaced or malicious plugin that omits compute processes will make a card look allocatable.
• The control plane listens on loopback by default. There is no authentication. X-ServerPilot-Actor is an audit label; any local process can send it, take the allocator role, create endpoints, change keepalive policy, claim GPUs, or name another actor and release that actor's leases. Local processes under the same user account are not isolated from each other. GPU UUID and endpoint are the resource identity boundary.

📚 Documentation

In English:

• Agent operating rules — the short contract you can paste into an agent's global rules
• Security · Contributing · Code of conduct · Changelog

In Chinese:

• Agent / MCP guide
• Observation and keepalive
• Upgrade checklist
• Implementation and verification status

Reference documentation is currently written in Chinese, and so are the desktop app and the descriptions an agent reads over MCP. This README, the changelog, and the agent operating rules are the English surfaces today.

License

MIT