diff --git a/docs/docs.json b/docs/docs.json index 2b99e176e..d012a7aa1 100644 --- a/docs/docs.json +++ b/docs/docs.json @@ -75,7 +75,8 @@ "group": "Integrations", "pages": [ "integrations/langgraph-integration", - "integrations/openenv-integration" + "integrations/openenv-integration", + "integrations/skypilot-integration" ] }, { diff --git a/docs/getting-started/installation-setup.mdx b/docs/getting-started/installation-setup.mdx index 25a1a0ada..2baf00da3 100644 --- a/docs/getting-started/installation-setup.mdx +++ b/docs/getting-started/installation-setup.mdx @@ -14,7 +14,7 @@ pip install openpipe-art ### Running the server locally -The ART server can be run locally on any machine with a GPU. To install the backend dependencies required for training and inference, you can install the `backend` extra: +The ART server can be run locally on any machine with a GPU. If you don't have one, the [SkyPilot integration](/integrations/skypilot-integration) can provision one on Kubernetes or any major cloud. To install the backend dependencies required for training and inference, you can install the `backend` extra: ```bash pip install openpipe-art[backend] diff --git a/docs/integrations/skypilot-integration.mdx b/docs/integrations/skypilot-integration.mdx new file mode 100644 index 000000000..bd8b11daa --- /dev/null +++ b/docs/integrations/skypilot-integration.mdx @@ -0,0 +1,79 @@ +--- +title: "SkyPilot" +description: "Provision a GPU for ART on Kubernetes or any cloud with SkyPilot" +--- + +# SkyPilot Integration + +`LocalBackend` needs a machine with a modern NVIDIA GPU. If you don't have one handy, [SkyPilot](https://github.com/skypilot-org/skypilot) can launch it for you on Kubernetes or any major cloud (AWS, GCP, Azure, and [more](https://docs.skypilot.co/en/latest/getting-started/installation.html)). This requires no ART-side code: `LocalBackend` runs unchanged on the machine SkyPilot provisions. + +## Setup + +Install SkyPilot with the infrastructure you use, then verify your credentials: + +```bash +pip install "skypilot[kubernetes]" # or [aws], [gcp], [azure], ... +sky check +``` + +## Define the machine + +Create `art.yaml` next to your training script: + +```yaml +resources: + accelerators: ["H100:1", "H200:1", "A100-80GB:1"] # tried in order + +workdir: . # sync this directory to the cluster + +envs: + PYTHONUTF8: "1" # avoids an encoding issue in unsloth-zoo + +setup: | + sudo apt-get update && sudo apt-get install -y python3.12-dev # needed by torch inductor + export PATH="$HOME/.local/bin:$PATH" # make sure uv is on PATH + uv venv --python 3.12 ~/art-venv + source ~/art-venv/bin/activate + # --torch-backend matches CUDA wheels; ninja is needed by flashinfer + uv pip install --torch-backend=auto "openpipe-art[backend]" ninja + +run: | + source ~/art-venv/bin/activate + python train.py +``` + +Your training script uses `LocalBackend` exactly as it would on any other machine: + +```python +from art.local import LocalBackend + +backend = LocalBackend(path="./.art") +``` + +## Launch + +```bash +sky launch -c art art.yaml +``` + +SkyPilot provisions the first available of the listed GPUs across your enabled infrastructure, syncs your working directory, and runs the script, streaming its output. On the first rollout, ART bootstraps a separate vLLM runtime environment, so expect a few quiet minutes in the logs before training output appears. From there: + +```bash +sky logs art # reattach to output (Ctrl-C detaches without killing the job) +ssh art # drop into the machine, e.g. to run in tmux +sky down art # tear it down when finished +``` + +`sky down` deletes the cluster's disk. With the `path="./.art"` above, checkpoints land in `.art/` inside the synced directory, so copy anything you want to keep first (`rsync -Pavz art:sky_workdir/.art ./`) or mount a bucket with [`file_mounts`](https://docs.skypilot.co/en/latest/reference/yaml-spec.html). + +To reach the vLLM or ART server from outside the cluster, add `ports: [7999, 8000]` under `resources` and get the URL with `sky status --endpoints art`. + +For iterating, keep the cluster up and rerun with `sky exec art art.yaml` after local edits; only the `run` section executes again. + +Environment variables like `WANDB_API_KEY` can be passed from your shell with `sky launch ... --secret WANDB_API_KEY`. + +## Further reading + +- [SkyPilot docs](https://docs.skypilot.co/) +- [Task YAML reference](https://docs.skypilot.co/en/latest/reference/yaml-spec.html) +- [Managed jobs](https://docs.skypilot.co/en/latest/examples/managed-jobs.html) for long-running training with automatic recovery diff --git a/docs/tutorials/summarizer.mdx b/docs/tutorials/summarizer.mdx index 031025208..40a3335c3 100644 --- a/docs/tutorials/summarizer.mdx +++ b/docs/tutorials/summarizer.mdx @@ -39,7 +39,7 @@ You'll be using `LocalBackend` to manage the GPU that your model will be trained pip install openpipe-art[backend] ``` -Make sure you have access to a machine with a modern NVIDIA GPU. This can be your local workstation or a cloud VM. If you're using a cloud provider (e.g. RunPod, Lambda, or GCP), launch the GPU instance and run the rest of this tutorial on that machine. +Make sure you have access to a machine with a modern NVIDIA GPU. This can be your local workstation or a cloud VM. If you're using a cloud provider (e.g. RunPod, Lambda, or GCP), launch the GPU instance and run the rest of this tutorial on that machine. [SkyPilot](/integrations/skypilot-integration) can launch one for you on Kubernetes or any major cloud. ### 3. Set up optional environment variables found in `.env.example`.