[nabinchha]

📋 Summary

Allow one vLLM replica to use tensor parallelism across multiple nodes, including TP16/PP1 with expert parallelism across four nodes with four GPUs each. Existing configurations keep their current TP-within-node / PP-across-node behavior when pipeline_parallel is omitted.

🔗 Related Issue

Related to #850; extends the serving topology delivered under #866.

🔄 Changes

• Add optional topology.pipeline_parallel; derive per-node GPU slices and replica counts from the TP × PP world size and replica node count.
• Separate node rank/count from pipeline parallelism in serving records, rendezvous, GPU ownership validation, and headless follower launch commands.
• Preserve existing serialized configs, plans, and serving goldens, including legacy process records.
• Allow expert parallelism within multi-node replicas and document the configuration and its distinction from coordinated data-parallel expert groups.

🧪 Testing

• Slurm suite: 2,101 passed, 1 platform-specific skip.
• Integration coverage for TP16/PP1+EP, multiple replica lanes/groups, hybrid TP/PP, single-node PP with TP1, invalid world geometry, and serialization round trips.
• Existing plan and serving golden bytes remain unchanged.
• Ruff, Fern validation, and internal links across 101 pages pass.
• Live Kimi K3 smoke with moonshotai/Kimi-K3 MXFP4 and vLLM 0.32.0 on two nodes with eight H100 GPUs each: all 16 TP/EP ranks loaded, readiness passed, two records generated, and the managed run exited successfully.
• This 16×H100 layout required 40 GiB of CPU offload per GPU and an increased request timeout. It is a functional smoke test, not a GPU-only performance result. Allocator warnings during weight preparation recovered before readiness.
• Collection preserved both records exactly across two Parquet partitions, using an explicit 64 GiB CPU-memory override in the test harness. Default collection submission inherited the GPU job's 1,500 GiB memory request and was rejected by the CPU partition; that existing resource-selection issue remains unfixed in this PR.
• This change implements global TP+EP; it does not add coordinated cross-node DP+EP mode.

✅ Checklist

• Follows commit message conventions
• Commits are signed off (DCO)
• User-facing topology and benchmark documentation updated
• Architecture docs: N/A; no standalone Slurm subsystem document is present.

[github-actions]

Fern preview: https://nvidia-preview-pr-1112.docs.buildwithfern.com/nemo/datadesigner

Fern previews include the docs-website version archive with PR changes synced into latest. Notebook tutorials are rendered without execution outputs in previews.

[greptile-apps]

Confidence Score: 5/5

[Medium impact] This PR appears safe to merge; no actionable issues were found.

Summary

This PR lets one vLLM replica spread tensor parallelism across nodes without tying the node count to pipeline parallelism.

• Deployment settings can spread tensor parallelism across replica nodes.
• Slurm launches cross-node replicas with separate node and pipeline ranks.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A["Authored TP, optional PP, and replica nodes"] --> B["Default omitted PP to replica nodes"]
  B --> C["Calculate GPUs per replica and per node"]
  C --> D["Check GPU slices and derive replica lanes"]
  D --> E["Resolve one process per replica node"]
  E --> F["Node rank 0: API server"]
  E --> G["Other node ranks: headless followers"]
  F --> H["Shared replica rendezvous"]
  G --> H

Loading

_{Reviews (1) · Last reviewed commit: "test(slurm): cover local PP with TP1" · Reviewed by Greptile}

[svcnvidia-nemo-ci]

Completed the light review across all 16 changed files and their affected planning, serving, runtime, and serialization paths. No confirmed Critical or Warning findings.

The topology integration tests cover cross-node TP/EP, hybrid TP/PP, single-node PP, legacy defaults, and invalid process geometry. Source inspection found the node/pipeline separation consistent across GPU slicing, rendezvous, head/follower commands, and legacy record serialization.

Static review only; tests, Slurm, and vLLM were not executed.

Note: formal approval is not available for this request, so this review is posted as a comment only.

Review generated by codex. This was a full review. Request a full review with /review reset=true.

[andreatnvidia]

Looks good to me. Validated TP8/PP1 + EP across two nodes, including generation and collection. The smoke used plain NCCL with optimized all-reduce paths disabled. No blocking findings.