nabinchha · Oct 30, 2026 · Oct 29, 2026 · Oct 29, 2026 · Oct 30, 2026
@@ -2,16 +2,50 @@
 
 Use a benchmark to compare concurrency and deployment topology while keeping each case as an ordinary Data Designer Slurm run. No benchmark controller stays resident after submission.
 
-Complete [Getting Started](/slurm/getting-started) first so the profile, images, and base run are ready.
+Complete [Getting Started](/slurm/getting-started) first so the profile, serving image, and builder config are ready.
+
+## Prepare a compatible base run
+
+This example requires two nodes with eight GPUs per node. Confirm that the selected profile or its detected GPU count requests eight GPUs per node. Reuse `builder.yaml` and the registered `vllm` image from Getting Started, and select a model that supports TP8 and fits in eight GPUs. The TP1 run from Getting Started cannot span two nodes with the default PP1.
+
+Create `benchmark-run.yaml` next to `builder.yaml`:
+
+```yaml
+schema_version: 1
+name: benchmark-base-run
+builder:
+  source: builder.yaml
+invocation:
+  num_records: 8
+  dataset_name: greetings
+  model_concurrency:
+    generator: 8
+deployments:
+  - model_alias: generator
+    served_model_name: example/generator
+    model: example/generator
+    server:
+      type: vllm
+      image:
+        name: vllm
+      startup_timeout: 15m
+      distributed_init_timeout: 10m
+    topology:
+      tensor_parallel: 8
+submission:
+  time_limit: 01:00:00
+```
+
+The repository ships this base run as `packages/data-designer-slurm/examples/benchmark-run.yaml`, alongside the matching `benchmark.yaml`. Keep PP omitted to use its default of one stage. For other GPU counts, choose a TP size that distributes evenly across both one-node and two-node groups and fits the GPUs available in each group.
 
 ## Define cases
 
-Create `benchmark.yaml` next to an existing `run.yaml`:
+Create `benchmark.yaml` next to `benchmark-run.yaml`:
 
 ```yaml
 schema_version: 1
 name: generator-scaling
-base_run: run.yaml
+base_run: benchmark-run.yaml
 model_aliases: all
 concurrency_values:
   - 32
@@ -38,7 +72,9 @@ The compiler expands the authored order deterministically. For this example, the
 
 Each benchmark case runs as one Slurm array task, even when the base run uses multiple tasks. The record policy sets the sample size for that task. This measures the throughput of one allocation, which analysis uses to estimate how many such tasks are needed for `target_total_records` within `target_runtime`.
 
-`nodes_per_replica` controls the number of nodes hosting each replica. When `pipeline_parallel` is omitted, it defaults to `nodes_per_replica`, and the resolved replica count is `(nodes / nodes_per_replica) * (gpus_per_node / tensor_parallel)`. With an explicit `pipeline_parallel`, each replica uses `tensor_parallel * pipeline_parallel` GPUs, distributed evenly across its nodes. Benchmark deployment cases change `nodes` and `nodes_per_replica`; they keep each deployment's `tensor_parallel` and explicit `pipeline_parallel` from the base run. Invalid combinations fail before submission. See [GPU topology and replica count](/slurm/run-configuration#gpu-topology-and-replica-count) for concrete eight-GPU examples.
+`nodes_per_replica` controls the number of nodes hosting each replica. `pipeline_parallel` defaults to `1`, independently of the node count. Each replica uses `tensor_parallel * pipeline_parallel` GPUs, distributed evenly across its nodes; the resolved replica count is `(nodes * gpus_per_node) / (tensor_parallel * pipeline_parallel)`. Benchmark deployment cases change `nodes` and `nodes_per_replica`; they keep each deployment's TP and PP sizes from the base run. Changing the node count does not add pipeline stages. Each child plan is validated before its submission. An invalid later case can leave earlier cases submitted, so choose a base topology compatible with every case. See [GPU topology and replica count](/slurm/run-configuration#gpu-topology-and-replica-count) for concrete eight-GPU examples.
+
+Changing only `nodes_per_replica` changes placement, not GPUs per replica or replica count. For example, with TP8, default PP1, and eight GPUs per node, both cases above create two eight-GPU replicas. The first places each replica on one node; the second spreads each replica across both nodes with four GPUs per node. Use a different base run to compare different TP or PP sizes.
 
 ## Run
 

@@ -91,7 +91,7 @@ For runs with local deployments, `gpus_per_node` is both the GPU count used for
 
 Multi-node generation jobs request exclusive nodes, including in `gres` mode. This keeps serving ports isolated because Slurm reports batch-job GPU IDs for the first node only. A partial-node `gpus_per_node` setting still limits the GPUs used per node, but other jobs cannot share those nodes during the run.
 
-The profile decides how many GPUs the job requests **per node**. Each deployment's `tensor_parallel`, optional `pipeline_parallel`, and `nodes_per_replica` decide how those GPUs form vLLM replicas. For example, one node with `gpus_per_node: 8` and `tensor_parallel: 1` runs eight one-GPU replicas; it does not request only one GPU. See [GPU topology and replica count](/slurm/run-configuration#gpu-topology-and-replica-count).
+The profile decides how many GPUs the job requests **per node**. Each deployment's `tensor_parallel`, `pipeline_parallel` (default: `1`), and `nodes_per_replica` decide how those GPUs form vLLM replicas. Increasing `nodes_per_replica` does not increase PP. For example, one node with `gpus_per_node: 8` and `tensor_parallel: 1` runs eight one-GPU replicas; it does not request only one GPU. See [GPU topology and replica count](/slurm/run-configuration#gpu-topology-and-replica-count).
 
 When selecting multiple partitions with `gpus_per_node: auto`, every selected partition must report GPU resources and all eligible nodes must report the same GPU count. Choose partitions that can also satisfy the run's GPU topology, model, and image requirements.
 

@@ -219,51 +219,88 @@ resources:
 topology:
   tensor_parallel: 8
   nodes_per_replica: 2
+  pipeline_parallel: 2
 ```
 
-In an eight-GPU profile, this starts two replicas, each spanning two nodes with 16 GPUs total. When `pipeline_parallel` is omitted, it defaults to `nodes_per_replica`, preserving this pipeline-parallel layout.
+In an eight-GPU profile, this starts two replicas, each spanning two nodes with 16 GPUs total. Each replica has two pipeline stages, each using TP8. TP communication stays within each node, and intermediate activations pass between the stages across nodes.
 
 ### GPU topology and replica count
 
-`tensor_parallel` and `pipeline_parallel` specify vLLM's tensor and pipeline parallel sizes for one replica. `nodes_per_replica` specifies how many nodes host that replica. Omitting `pipeline_parallel` defaults it to `nodes_per_replica`, so existing configurations continue to use tensor parallelism within each node and pipeline parallelism across nodes.
+One replica serves requests using a group of GPU workers. Choose how those workers cooperate:
 
-To spread tensor parallelism across nodes while keeping PP=1, set `pipeline_parallel: 1` explicitly. For a profile requesting **four GPUs per node**, this configuration runs one TP16/PP1 replica across four nodes, with expert parallelism enabled:
+- **Tensor parallelism (TP)** splits work within model layers across GPUs. A TP group can span multiple nodes.
+- **Pipeline parallelism (PP)** splits model layers into sequential stages. A replica can use TP, PP, or both.
+- **Independent replicas** serve different requests with separate copies of the model. The planner derives their count from the GPU budget and the topology of each replica.
+
+`tensor_parallel` and `pipeline_parallel` specify the TP and PP sizes for one replica. `nodes_per_replica` specifies how many nodes host that replica; it does not select the parallelism method.
+
+<Note>
+`pipeline_parallel` defaults to `1`, regardless of `nodes_per_replica`. Spreading a replica across nodes does not add pipeline stages. Set `pipeline_parallel: 2` explicitly when you want two stages. TP and PP do not need to be equal; their product determines the GPU count per replica.
+</Note>
+
+For a profile requesting **four GPUs per node**, this configuration runs one TP16 replica across four nodes. PP defaults to 1, and expert parallelism defaults to disabled:
 
 ```yaml
 resources:
   nodes: 4
 topology:
   tensor_parallel: 16
-  pipeline_parallel: 1
   nodes_per_replica: 4
 server:
   type: vllm
   image:
     name: my-serving-image
-  enable_expert_parallel: true
 ```
 
-The serving image must support the selected model and parallel settings. The runtime starts one vLLM process on each replica node, with one API server and headless followers sharing a rendezvous. EP operates within each replica; independent replica lanes do not form a coordinated data-parallel expert group.
+The serving image must support the selected model and parallel settings. The runtime starts one vLLM process on each replica node, with one API server and headless followers sharing a rendezvous.
+
+### One TP16 replica across two eight-GPU nodes
+
+For a profile requesting **eight GPUs per node**, the following deployment fragment creates one replica using all 16 GPUs:
+
+```yaml
+resources:
+  nodes: 2
+topology:
+  tensor_parallel: 16
+  nodes_per_replica: 2
+```
+
+Node 0 hosts the API server and eight GPU workers; node 1 hosts a headless follower and eight GPU workers. Both nodes cooperate on the same requests. The native Data Designer client uses a separate CPU step and sends requests to the replica through its logical endpoint.
+
+This topology describes GPU placement, not whether the model fits in memory. Check the serving runtime's model, quantization, memory, and interconnect requirements separately. If you configure CPU offloading through `server.extra_args`, allow enough host memory and increase model request timeouts to account for the additional latency.
+
+### Optional expert parallelism
+
+For a Mixture-of-Experts (MoE) model, `server.enable_expert_parallel: true` asks vLLM to distribute routed experts across its expert-parallel ranks instead of using tensor parallelism to split each expert. Attention layers still use the configured TP layout. See [vLLM's expert parallelism guide](https://docs.vllm.ai/en/latest/serving/expert_parallel_deployment/) for the layer behavior and backend requirements.
+
+EP defaults to `false` and is not required for a TP group to span nodes. Enabling it does not multiply the GPU count: the TP16/PP1 example still uses 16 GPUs in one replica. Choose EP based on the model, serving image, communication backend, and measured workload behavior. The planner does not create coordinated data-parallel expert groups across independent replicas.
+
+### Calculate replica counts
 
 The planner distributes each replica's GPU workers evenly across its nodes:
 
 ```text
-PP = pipeline_parallel, or nodes_per_replica when omitted
+PP = pipeline_parallel (default: 1)
 GPUs per replica = tensor_parallel * PP
 GPUs per replica node = GPUs per replica / nodes_per_replica
 replicas = (resources.nodes / nodes_per_replica) * (gpus_per_node / GPUs per replica node)
 ```
 
-For a profile that requests eight GPUs per node, the following are separate choices for **one deployment in one array task**, with `pipeline_parallel` omitted:
+For a profile that requests eight GPUs per node, the following are separate choices for **one deployment in one array task**. PP is explicit in this table:
+
+| `resources.nodes` | TP | PP | `nodes_per_replica` | vLLM replicas | GPUs per replica |
+| ---: | ---: | ---: | ---: | ---: | ---: |
+| 1 | 1 | 1 | 1 | 8 | 1 |
+| 1 | 2 | 4 | 1 | 1 | 8 |
+| 2 | 8 | 1 | 1 | 2 | 8 |
+| 2 | 8 | 2 | 2 | 1 | 16 |
+| 2 | 16 | 1 | 2 | 1 | 16 |
+| 2 | 4 | 4 | 2 | 1 | 16 |
+| 4 | 16 | 1 | 2 | 2 | 16 |
+| 4 | 1 | 1 | 1 | 32 | 1 |
 
-| `resources.nodes` | `tensor_parallel` | `nodes_per_replica` | vLLM replicas | GPUs per replica |
-| ---: | ---: | ---: | ---: | ---: |
-| 1 | 1 | 1 | 8 | 1 |
-| 4 | 1 | 1 | 32 | 1 |
-| 4 | 2 | 1 | 16 | 2 |
-| 4 | 4 | 1 | 8 | 4 |
-| 4 | 8 | 1 | 4 | 8 |
-| 4 | 8 | 2 | 2 | 16 |
+These are valid planner layouts; each model and serving image must also support the selected TP and PP sizes. In the four-node TP16/PP1 example, each pair of nodes hosts an independent 16-GPU replica. Enabling EP keeps those two replicas separate.
 
 Increasing `resources.nodes` at TP=1 creates more independent one-GPU servers; it does not combine their GPUs into one larger server. The client uses one logical endpoint per model alias, backed by a local proxy that routes requests among that alias's replicas. `invocation.model_concurrency` limits in-flight model calls **per alias within each array task**; it does not set the replica count or the number of array tasks.
 

@@ -0,0 +1,23 @@
+schema_version: 1
+name: benchmark-base-run
+builder:
+  source: builder.yaml
+invocation:
+  num_records: 8
+  dataset_name: greetings
+  model_concurrency:
+    generator: 8
+deployments:
+  - model_alias: generator
+    served_model_name: example/generator
+    model: example/generator
+    server:
+      type: vllm
+      image:
+        name: vllm
+      startup_timeout: 15m
+      distributed_init_timeout: 10m
+    topology:
+      tensor_parallel: 8
+submission:
+  time_limit: 01:00:00
@@ -1,6 +1,6 @@
 schema_version: 1
 name: generator-scaling
-base_run: run.yaml
+base_run: benchmark-run.yaml
 model_aliases: all
 concurrency_values:
   - 32

@@ -275,7 +275,7 @@ def validate_shared_node(self) -> DeploymentResources:
 class DeploymentTopology(AuthoredConfig):
     tensor_parallel: PositiveInt = 1
     nodes_per_replica: PositiveInt = 1
-    pipeline_parallel: PositiveInt | None = Field(default=None, exclude_if=lambda value: value is None)
+    pipeline_parallel: PositiveInt = Field(default=1, exclude_if=lambda value: value == 1)
 
 
 class ServerDeploymentConfig(AuthoredConfig):

@@ -265,14 +265,13 @@ def derive(
         gpus_per_node: int,
         tensor_parallel: int,
         nodes_per_replica: int,
-        pipeline_parallel: int | None = None,
+        pipeline_parallel: int = 1,
     ) -> ResolvedTopology:
         """Derive the canonical v1 topology from placement and GPU resources."""
         if min(node_count, gpus_per_node, tensor_parallel, nodes_per_replica) <= 0:
             raise ValueError("topology derivation inputs must be positive")
         if node_count % nodes_per_replica:
             raise ValueError("nodes_per_replica must divide the deployment node count")
-        pipeline_parallel = nodes_per_replica if pipeline_parallel is None else pipeline_parallel
         if pipeline_parallel <= 0:
             raise ValueError("pipeline_parallel must be positive")
         gpus_per_replica = tensor_parallel * pipeline_parallel

@@ -3,10 +3,109 @@
 
 from __future__ import annotations
 
+import re
+from pathlib import Path
+
 import pytest
+import yaml
 
 from data_designer.slurm.benchmark import BenchmarkCompiler
-from data_designer.slurm.config import DataDesignerSlurmBenchmarkConfig, DataDesignerSlurmConfig
+from data_designer.slurm.config import (
+    DataDesignerSlurmBenchmarkConfig,
+    DataDesignerSlurmConfig,
+    load_benchmark_config,
+    load_run_config,
+)
+from data_designer.slurm.planning import ResolvedDependencyLock, ResolvedSlurmRunPlan
+from data_designer.slurm.planning.compiler import SlurmRunCompiler
+from data_designer.slurm.planning.resolution import resolve_slurm_config
+
+
+@pytest.mark.parametrize("source", ["shipped", "guide"])
+def test_benchmark_examples_compile_every_child_plan(
+    source: str,
+    tmp_path: Path,
+    dependency_lock_single: ResolvedDependencyLock,
+    single_node_plan: ResolvedSlurmRunPlan,
+) -> None:
+    examples = Path(__file__).parents[2] / "examples"
+    benchmark_file = examples / "benchmark.yaml"
+    if source == "guide":
+        guide = Path(__file__).parents[4] / "fern/versions/latest/pages/slurm/benchmarks.mdx"
+        blocks = re.findall(r"```yaml\n(.*?)```", guide.read_text(), flags=re.DOTALL)
+        assert len(blocks) == 2
+        (tmp_path / "benchmark-run.yaml").write_text(blocks[0])
+        benchmark_file = tmp_path / "benchmark.yaml"
+        benchmark_file.write_text(blocks[1])
+    config = load_benchmark_config(benchmark_file)
+    assert config.base_run.source is not None
+    base_run = load_run_config(benchmark_file.parent / config.base_run.source)
+    assert base_run.builder.source is not None
+    builder_payload = yaml.safe_load((examples / base_run.builder.source).read_text())
+    compiled = BenchmarkCompiler.compile(config, base_run)
+    image = single_node_plan.deployments[0].image.model_copy(
+        update={"authored_ref": base_run.deployments[0].server.image}
+    )
+
+    assert len(compiled.cases) == 4
+    for case in compiled.cases:
+        plan = SlurmRunCompiler.compile(
+            resolve_slurm_config(
+                case.child_run_config,
+                selected_profile=single_node_plan.selected_profile,
+                client_runtime=single_node_plan.client.runtime,
+                deployment_images=(image,),
+                dependency_lock=dependency_lock_single,
+                runtime_bundle=single_node_plan.runtime_bundle,
+                run_id=case.child_run_id,
+                package_version=single_node_plan.package_version,
+                resolved_gpus_per_node=8,
+                builder_payload=builder_payload,
+            )
+        )
+        topology = plan.deployments[0].topology
+        assert topology.tensor_parallel == 8
+        assert topology.pipeline_parallel == 1
+        assert topology.gpus_per_replica == 8
+        assert topology.replica_count == 2
+        assert topology.nodes_per_replica == case.child_run_config.deployments[0].topology.nodes_per_replica
+
+
+@pytest.mark.parametrize("pipeline_parallel", [None, 2], ids=["default-pp1", "explicit-pp2"])
+def test_benchmark_node_placement_preserves_pipeline_size_and_replica_count(
+    benchmark_config: DataDesignerSlurmBenchmarkConfig,
+    authored_run_single: DataDesignerSlurmConfig,
+    dependency_lock_single: ResolvedDependencyLock,
+    single_node_plan: ResolvedSlurmRunPlan,
+    pipeline_parallel: int | None,
+) -> None:
+    payload = authored_run_single.model_dump(mode="json")
+    topology = payload["deployments"][0]["topology"]
+    topology["tensor_parallel"] = 8 if pipeline_parallel is None else 4
+    topology.pop("pipeline_parallel", None)
+    if pipeline_parallel is not None:
+        topology["pipeline_parallel"] = pipeline_parallel
+    base_run = DataDesignerSlurmConfig.model_validate(payload)
+    compiled = BenchmarkCompiler.compile(benchmark_config, base_run)
+
+    for case in compiled.cases:
+        plan = SlurmRunCompiler.compile(
+            resolve_slurm_config(
+                case.child_run_config,
+                selected_profile=single_node_plan.selected_profile,
+                client_runtime=single_node_plan.client.runtime,
+                deployment_images=(single_node_plan.deployments[0].image,),
+                dependency_lock=dependency_lock_single,
+                runtime_bundle=single_node_plan.runtime_bundle,
+                run_id=case.child_run_id,
+                package_version=single_node_plan.package_version,
+                resolved_gpus_per_node=8,
+            )
+        )
+        resolved = plan.deployments[0].topology
+        assert resolved.pipeline_parallel == (1 if pipeline_parallel is None else pipeline_parallel)
+        assert resolved.gpus_per_replica == 8
+        assert resolved.replica_count == 2
 
 
 def test_compiler_expands_stable_ordered_ordinary_runs(