Repository navigation
Let a tong request host GPUs via resources.gpus - #92
Merged
Merged
Conversation
A tong could not reach the host's GPUs, so a GPU-backed service such as a model server could not run as one. `resources.gpus` now maps to `docker run --gpus`. The value is held to a strict grammar rather than passed through free form: `all`, a count from 1 to 2^31-1 (an int or ASCII digits), `device=<id>`, or the docker-quoted list `"device=<id>,<id>,..."`. Docker's `driver=`, `capabilities=`, and `options=` keys are refused because they grant more than GPU access -- `driver=cdi` injects any registered device -- and `count=` because a plain count says the same. A free-form string could also carry control bytes that rewrite the approval prompt yet still launch; no such value passes validation now. An unquoted `device=0,1`, the likeliest mistake since docker reads the value as CSV, gets an error that shows the quoting it needs. Validation and argv assembly both go through `parse_gpus`, so the two cannot disagree about what is accepted, and a bad value stops the launch before docker runs. The config hash already covers the whole definition, so adding or changing a GPU request recreates a running `shared` tong.
GPU access is a grant of host devices, so a workspace-sourced tong that asks for it must say so before the user approves it. The privilege summary now carries `resources.gpus` and the prompt renders it on its own line; a tong that asks for none shows nothing. The gate runs before validation, so the summary reports the value as declared rather than parsed: whatever could reach docker is what the user sees. Approval is keyed by the definition hash, so adding a GPU request to an approved tong re-prompts.
Describe `resources.gpus` next to `memory`: the forms it accepts, the literal double quotes docker's CSV parsing needs around a device list, why docker's other `--gpus` keys are refused, the GPU-capable docker host it needs, and that a workspace tong's request shows in the approval prompt.
This was referenced Oct 4, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A tong could not ask for GPU access, so a model server like Ollama could not run as a tong with the GPU support
make run_ollamagives it today (--gpus=all). This addsresources.gpus, passed todocker run --gpus, so that tong can be built outside this repo. The existing Ollama make targets are unchanged.What changes
resources.gpus: a new field, parsed by one helper (parse_gpusinswarmforge/tongs/model.py) that both validation and argv assembly call, so the two cannot disagree. Only these forms are accepted:alldevice=<id>, or the docker-quoted list"device=<id>,<id>", where each id starts with a letter or digit and continues with letters, digits, and._:-Docker's other
--gpuskeys are refused.driver=,capabilities=andoptions=can grant more than GPU access (driver=cdiinjects any device registered on the host).count=is redundant with a plain count. Leaving--gpusfree-form would also have let control characters reach docker. An unquoteddevice=0,1, the likeliest mistake given docker's CSV parsing, is refused with the exact YAML quoting to use instead.Approval gate: GPU access is a host-device grant, so the privilege summary reports it and a workspace tong's approval prompt shows a
gpus: <value> (host GPU access)line. The config hash already covers the whole definition, so adding or changinggpusasks for approval again and recreates a runningsharedtong. Tests cover both.Docs: a new Resources section in
docs/tongs/definitions.mdcovers the accepted forms, the quoting, why the other keys are refused, and the host requirement. GPU containers need the NVIDIA Container Toolkit on Linux; Docker Desktop's WSL 2 backend provides GPU support itself. Without itdocker runfails and the launch stops.docs/tongs/approval.mdlists GPU access among what the prompt shows.Testing
make test: 948 tests pass (923 on master). Each commit also passes on its own.ruff checkis clean.--gpuskeys), plus the--gpusflag placement before the image in the tong'sdocker runargv and through the orchestrator, the approval prompt line, and a non-mappingresourcesvalue.docker run --gpusagainst a GPU host. That needs a host with NVIDIA tooling. The quickest manual check is a user-layersharedtong withresources: {gpus: all}runningnvidia-smi.