[kh0pper]

Follow-up (a) to #508, recorded in queue item 30. Stacked on #509, so merge #509 first. Once #509 lands, this branch is rebased onto main, so only its own commits remain.

The problem. servers/gateway/gpu-orchestrator.js started model bundles with docker compose up from the repo's compose file. That path skipped the install security check (compose-guard.js), and it passed the gateway's whole environment, secrets included, to Compose.

What changes

• The orchestrator runs only model bundles. These are bundles whose repo manifest says inference: true and names the same id. A provider row synced from the fleet can name any id, so the row alone isn't trusted.
• One invocation for the check and the start. Every up is checked first by the shared model check. The check sees the exact model that up will apply: the same -f file, the same working folder, and the gateway's allowlisted compose env (no secrets, no COMPOSE_*). stop uses the same invocation.
• The check runs before eviction. It happens before any mutex sibling is stopped, so a refused start never costs the box its resident model. A refusal throws ComposeGuardError and nothing starts.
• A manifest declares what a model bundle needs. The new host_grants field lists gpu, shm_size, env_files and path_env. It is read from the repo manifest only, never from a community bundle or an installed copy.
  • GPU devices, gpus, GPU reservations, runtime: nvidia and NVIDIA_* env need host_grants.gpu.
  • A /dev/shm above 1 GiB needs host_grants.shm_size.
  • An env_file outside the bundle is allowed only if it is declared and sits in ~/.crow/env. inference: true no longer opens any host file by itself.
  • The folders named by path_env keys (for example LLM_CACHE) come from the bundle's .env in the checkout and are passed as the check's operator roots. Since fix(bundles): outside Crow's home, extensions mount only folders the owner allowed #524, a mount outside Crow's home must sit under one of them, and the floor still applies: system folders, home dot-folders and ~/bin, a git checkout, and anything containing Crow's home stay refused.
• The env_file checks fail closed.
  • Any NVIDIA_ mention in an admitted env_file needs the gpu grant. This covers any case, separator, quoting or line ending.
  • The file is checked before it is opened, so a device or FIFO is never opened. Only a regular file is read, up to 1 MiB.
  • A missing env_file is refused unless it is marked required: false.
  • An env_file under a read-write bind of the same project is refused, because a container could swap it before Compose reads it.
• More bundles drop ipc: host. The 4b, embed, vllm-cuda-embed, vllm-cuda-rerank and vllm-cuda-vision bundles drop it and keep their private /dev/shm at shm_size. Each is one container, so the host IPC namespace only matters for sharing with other containers, and there are none. fix(models): the 35B bundle drops ipc: host (follow-up to #508) #509 drops the 35B's.
• composeEnv moves to compose-guard.js. The routes keep a thin wrapper.
• Versions and grants:
  • 4b, embed and the vllm-cuda-* bundles: 1.0.1.
  • 35B: 1.0.2 (on top of fix(models): the 35B bundle drops ipc: host (follow-up to #508) #509's 1.0.1; grants only).
  • sdxl and vllm: 1.0.2.
  • ollama and localai: 1.0.1.
  • The docs (en/es) list host_grants.

⚠ Merge inside a scheduled model window.

• This PR changes the 4b and embed compose files. Once it is pulled, the next orchestrator up of either container recreates it. That happens at their next start, not at merge time: a resident model is never re-upped.
• So, after merging, check in the window that both come up healthy:
  • 4b answers a completion;
  • embed answers an embedding with 1024 dims.
• The gateway needs a restart to run the new orchestrator code.
• A read-only check on a live host (docker compose config only) ran this branch's check on the three crow model bundles with that host's real .env values. All three pass and come out with a private IPC namespace.

Tests

• tests/compose-guard.test.js: grants, env_file handling, and every repo model bundle passing the check as the orchestrator runs it.
• New tests/gpu-orchestrator-compose-guard.test.js: covers these cases.
  • A refused bundle is never started.
  • A refused target doesn't evict the resident sibling.
  • The orchestrator runs only inference: true bundles.
  • The compose env carries no gateway secrets.
• Full suite locally: 10432 tests, 10397 pass, 0 fail, 35 skipped. Registry rebuilt; check-port-allocation passes.

Not in this PR

• Item 30b, the host-path allowlist, shipped in fix(bundles): outside Crow's home, extensions mount only folders the owner allowed #524.
• The installed copies under the Crow home keep their old compose files until a reinstall, because a refresh doesn't copy compose files. Those copies only matter for the Extensions Start route.

https://claude.ai/code/session_01N9fXzvNy8rCEArj1y5p6fp