Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion docs/docs.json
Original file line number Diff line number Diff line change
Expand Up @@ -75,7 +75,8 @@
"group": "Integrations",
"pages": [
"integrations/langgraph-integration",
"integrations/openenv-integration"
"integrations/openenv-integration",
"integrations/skypilot-integration"
]
},
{
Expand Down
2 changes: 1 addition & 1 deletion docs/getting-started/installation-setup.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ pip install openpipe-art

### Running the server locally

The ART server can be run locally on any machine with a GPU. To install the backend dependencies required for training and inference, you can install the `backend` extra:
The ART server can be run locally on any machine with a GPU. If you don't have one, the [SkyPilot integration](/integrations/skypilot-integration) can provision one on Kubernetes or any major cloud. To install the backend dependencies required for training and inference, you can install the `backend` extra:

```bash
pip install openpipe-art[backend]
Expand Down
79 changes: 79 additions & 0 deletions docs/integrations/skypilot-integration.mdx
Original file line number Diff line number Diff line change
@@ -0,0 +1,79 @@
---
title: "SkyPilot"
description: "Provision a GPU for ART on Kubernetes or any cloud with SkyPilot"
---

# SkyPilot Integration

`LocalBackend` needs a machine with a modern NVIDIA GPU. If you don't have one handy, [SkyPilot](https://github.com/skypilot-org/skypilot) can launch it for you on Kubernetes or any major cloud (AWS, GCP, Azure, and [more](https://docs.skypilot.co/en/latest/getting-started/installation.html)). This requires no ART-side code: `LocalBackend` runs unchanged on the machine SkyPilot provisions.

## Setup

Install SkyPilot with the infrastructure you use, then verify your credentials:

```bash
pip install "skypilot[kubernetes]" # or [aws], [gcp], [azure], ...
sky check
```

## Define the machine

Create `art.yaml` next to your training script:

```yaml
resources:
accelerators: ["H100:1", "H200:1", "A100-80GB:1"] # tried in order

workdir: . # sync this directory to the cluster

envs:
PYTHONUTF8: "1" # avoids an encoding issue in unsloth-zoo

setup: |
sudo apt-get update && sudo apt-get install -y python3.12-dev # needed by torch inductor
export PATH="$HOME/.local/bin:$PATH" # make sure uv is on PATH
uv venv --python 3.12 ~/art-venv
source ~/art-venv/bin/activate
# --torch-backend matches CUDA wheels; ninja is needed by flashinfer
uv pip install --torch-backend=auto "openpipe-art[backend]" ninja

run: |
source ~/art-venv/bin/activate
python train.py
```

Your training script uses `LocalBackend` exactly as it would on any other machine:

```python
from art.local import LocalBackend

backend = LocalBackend(path="./.art")
```

## Launch

```bash
sky launch -c art art.yaml
```

SkyPilot provisions the first available of the listed GPUs across your enabled infrastructure, syncs your working directory, and runs the script, streaming its output. On the first rollout, ART bootstraps a separate vLLM runtime environment, so expect a few quiet minutes in the logs before training output appears. From there:

```bash
sky logs art # reattach to output (Ctrl-C detaches without killing the job)
ssh art # drop into the machine, e.g. to run in tmux
sky down art # tear it down when finished
```

`sky down` deletes the cluster's disk. With the `path="./.art"` above, checkpoints land in `.art/` inside the synced directory, so copy anything you want to keep first (`rsync -Pavz art:sky_workdir/.art ./`) or mount a bucket with [`file_mounts`](https://docs.skypilot.co/en/latest/reference/yaml-spec.html).

To reach the vLLM or ART server from outside the cluster, add `ports: [7999, 8000]` under `resources` and get the URL with `sky status --endpoints art`.

For iterating, keep the cluster up and rerun with `sky exec art art.yaml` after local edits; only the `run` section executes again.

Environment variables like `WANDB_API_KEY` can be passed from your shell with `sky launch ... --secret WANDB_API_KEY`.

## Further reading

- [SkyPilot docs](https://docs.skypilot.co/)
- [Task YAML reference](https://docs.skypilot.co/en/latest/reference/yaml-spec.html)
- [Managed jobs](https://docs.skypilot.co/en/latest/examples/managed-jobs.html) for long-running training with automatic recovery
2 changes: 1 addition & 1 deletion docs/tutorials/summarizer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ You'll be using `LocalBackend` to manage the GPU that your model will be trained
pip install openpipe-art[backend]
```

Make sure you have access to a machine with a modern NVIDIA GPU. This can be your local workstation or a cloud VM. If you're using a cloud provider (e.g. RunPod, Lambda, or GCP), launch the GPU instance and run the rest of this tutorial on that machine.
Make sure you have access to a machine with a modern NVIDIA GPU. This can be your local workstation or a cloud VM. If you're using a cloud provider (e.g. RunPod, Lambda, or GCP), launch the GPU instance and run the rest of this tutorial on that machine. [SkyPilot](/integrations/skypilot-integration) can launch one for you on Kubernetes or any major cloud.

### 3. Set up optional environment variables found in `.env.example`.

Expand Down