未来能支持细粒度PP吗

[li-yr]

如题，现在PP还是通过公式估计的，之后能支持细粒度的估计吗

[tianhao909]

【人工回复】【Manual reply】
hi 这些是社区关于pp的进展的讨论

#163
#98
#18

[tianhao909]

[Reply from Agent A] [For reference only]

CN

Issue #283 回复（中文版）

原 Issue：aliyun/SimAI#283 — 未来能支持细粒度 PP 吗
提问者：@li-yr
提问内容：「如题，现在 PP 还是通过公式估计的，之后能支持细粒度的估计吗」

感谢提问。下面按你提的疑问，结合代码逐条分析并给出最小可行方案（Minimum Viable Plan, MVP）。

---

术语速览（中英对照）

英文术语	中文	含义
Pipeline Parallelism (PP)	流水线并行	把模型按层切到不同 rank，micro-batch 间用 P2P send/recv 串接
Tensor Parallelism (TP)	张量并行	单个矩阵乘按维度切分到多卡
Data Parallelism (DP)	数据并行	不同 rank 处理不同 mini-batch，梯度做 AllReduce
Expert Parallelism (EP)	专家并行	MoE 模型中专家分布到不同 rank
Virtual Pipeline Parallelism (VPP)	虚拟流水线并行	Interleaved-1F1B 中每个 rank 持有多个不连续的 chunk
Gradient Accumulation (GA)	梯度累积	一次迭代内累积若干 micro-batch 的梯度后再更新
Micro-batch	微批次	PP 调度的最小单位，把一个 mini-batch 拆成 GA 份
1F1B（One-Forward-One-Backward）	一前一后	稳态下交替 fwd/bwd 的经典 PP 调度
Bubble	流水线气泡	因依赖等待造成的 GPU 空闲时间
Closed-form formula	闭式公式	用解析式直接给出结果的算法
Point-to-Point (P2P)	点到点通信	单 sender → 单 receiver 的通信模式（如 send/recv）

---

一、需求背景总结

你看到的现象是对的：当前 SimAI 的 PP 时间确实是用两条闭式公式后处理估出来的，并没有进入仿真器事件流。

具体而言：

• TP / DP / EP 的集合通信通过 MockNcclGroup + NCCL Flow Model 真实生成 SingleFlow，注入 ns-3-alibabacloud 网络模型
• 但 PP 通信和 PP 气泡是在所有 layer 仿真完成后，由 Layer.cc 用两条公式直接叠加到 total_time 里

所以「细粒度 PP（Fine-grained PP Modeling）」=「让 PP 也走真实事件流，让气泡由仿真自然产生」是一个合理且必要的演进方向。

---

二、涉及文件 / 模块

序号	路径（GitHub 链接）	当前角色	改造点
1	workload/Workload.hh#L76-L83	全局 PP/vpp/GA/pp_commsize 字段	新增 stage 切分、调度策略字段
2	workload/Workload.cc#L1180-L1268	解析 workload 首行（pp: vpp: ga: pp_comm:）	加 pp_schedule: stage_layers: 解析
3	workload/Layer.cc#L504-L507 ＆ #L734-L737	闭式公式后处理（核心问题点）	细粒度模式下停用
4	system/Sys.cc#L1355-L1397	mock_nccl_grobal_group_init / mock_nccl_comms_init 中 PP_size = 1 硬编码	用真实 PP_size，并建 PP comm
5	system/Sys.hh#L49-L56	enum ParallelStrategy { TP, DP, PP, EP, DP_EP, NONE } 已预留但无人引用	接入 send/recv 调度
6	system/MockNcclGroup.h#L42-L49 ＆ .cc	GroupType::PP 已预留但无邻居映射	增加 PP 拓扑 + P2P SingleFlow 生成
7	system/AstraParamParse.hh#L60 ＆ #L156-L158	pp_overlap_ratio / --pp_overlap	加 --pp_mode --pp_schedule
8	aicb 仓库	生成 workload 文件	输出 stage 切分与 micro-batch 信息

---

三、根因分析 / 实现思路

3.1 当前公式法（formula 模式）

源码片段（Layer.cc#L504-L507）：

Tick Expose_PP_time = (2 * vpp * GA *
                       (pp_commsize * GBps / param->net_work_param.pp_overlap_ratio * 1e9)
                       / FREQ);
Expose_PP_time *= (1 - param->net_work_param.pp_overlap_ratio);
// pp bubble time
pre_bubble_time *= static_cast<double>(PP_size - 1) / (GA * vpp);

数学表达：

$$ \text{Expose_PP_time} ;=; 2 \cdot \text{vpp} \cdot \text{GA} \cdot \frac{\text{pp_commsize}}{\text{pp_overlap_ratio}} \cdot (1 - \text{pp_overlap_ratio}) $$

$$ \text{Bubble} ;=; \text{step_time} \cdot \frac{\text{PP_size} - 1}{\text{GA} \cdot \text{vpp}} $$

含义解读：

• 第一行：双向 P2P 通信总量 × 用户给的「不重叠比例」 → 估算暴露的 PP 通信时间
• 第二行：经典 1F1B 气泡公式（GPipe 论文给出的 $\frac{p-1}{m}$ 形式，按 vpp 进一步缩小）

根本问题：这两条都是事后乘除，完全脱离仿真器事件流，所以无法刻画下面这些真实效应：

真实效应	公式法表现
micro-batch 粒度的依赖（recv → fwd → send / recv → bwd → send）	完全不感知
PP P2P 与 TP/DP/EP 共享同一 NIC / 链路时的争用	无
1F1B / Interleaved-1F1B / ZB-H1 / ZB-2P / Chimera 不同调度气泡差异	全部只有同一公式
异构 layer 切分（uneven stage）→ stage 间负载不均	看不到
跨机架 / Pod 真实跳数与队列拥塞	无

3.2 细粒度方案（fine 模式）思路

graph TB
    A[AICB 生成 workload<br/>含 stage_layers + pp_schedule] --> B[Workload 解析 PP 调度参数]
    B --> C[PipelineScheduler<br/>按 1F1B 产出事件序列]
    C --> D{事件类型}
    D -->|fwd_compute| E[现有 Layer 计算路径]
    D -->|bwd_compute| E
    D -->|pp_send| F[Sys::sim_send<br/>走 NCCL Flow Model]
    D -->|pp_recv| G[Sys::sim_recv<br/>走 NCCL Flow Model]
    F --> H[ns-3 真实拥塞排队]
    G --> H
    E --> I[Tick 推进<br/>气泡=stage 自然等待时间]
    H --> I
    I --> J[Total Time 中气泡自然涌现]

Loading

关键点：fine 模式下气泡 = stage 间自然等待（无需后处理）；PP P2P 与 TP/DP/EP 共用同一 ns-3 拓扑，争用自然涌现。

---

四、最小可行三阶段修改方案（向后兼容）

为了「最小改动 + 不引入额外依赖 + 保持现有代码风格」，我建议分三阶段推进：

阶段 A：公式法精度修正（≤ 50 行，立刻可合）

改动	位置	内容
A1	Sys.cc:1363, 1376	int PP_size = 1; → int PP_size = workload->pipeline_model_parallelism; （当前 bug：DP_size 没除以 PP，导致 DP 通信组规模算大）
A2	Layer.cc:504, 734	加除零保护：pp_overlap_ratio == 0 时退化为 0
A3	AstraParamParse.hh:60	注释中明确语义：pp_overlap_ratio = 1 表示「100% 重叠 → 暴露=0」，README 标注

阶段 A 可作为对本 issue 的短期回应单独 PR。

阶段 B：PP P2P 接入仿真（≈ 200-400 行，加开关向后兼容）

改动	位置	内容
B1	AstraParamParse.hh	加 --pp_mode formula|fine（默认 formula）
B2	Sys.cc::mock_nccl_grobal_group_init	用真实 PP_size 创建 MockNccl::GroupType::PP 组，记录每 rank 的上下游邻居
B3	MockNcclGroup.cc	新增 generate_pp_p2p_flow(src, dst, size)，复用 SingleFlow
B4	Workload.cc	在每个 layer 完成时若位于 stage 边界且 pp_mode=fine，调用 sim_send/sim_recv（包大小 pp_commsize / GA）
B5	Layer.cc:504-507, 734-737	pp_mode=fine 时不再做闭式 Expose_PP_time 后处理

阶段 C：完整 PP 调度器（≈ 500-800 行，独立子模块）

改动	位置	内容
C1	新增 astra-sim/workload/PipelineScheduler.{hh,cc}	抽象基类 + Schedule1F1B / ScheduleInterleaved（用 vpp）/ ScheduleZeroBubble（接口预留）
C2	Workload.cc 主循环	由 scheduler 推进 micro-batch 事件序列
C3	aicb 仓库	workload 首行新增 pp_schedule: 1f1b|interleaved、stage_layers: …
C4	--pp_schedule CLI 参数	透传到 Workload

推荐节奏：先合阶段 A 修正现有公式法的 bug，B、C 列入 Roadmap。

---

五、测试前置条件

阶段	是否需要新数据	详情
A	否	复用现有 SimAI/example/microAllReduce.txt、workload_analytical.txt；新增一个 TP=2 / PP=2 / DP=2 的 8 卡回归用例验证 DP_size 修正后值不变
B	是（中等）	AICB 重新生成 workload，确保首行 pp_comm: 与真实 P2P 字节数一致；网络拓扑用现有 inputs/topo/Spectrum-X_*.txt
C	是（高）	需要 Megatron-LM 真实训练日志（forward / backward / bubble 时长）作为校准基线；至少 7B + 70B 两组对比样本

---

六、README 是否需要修改

需要，最小修改集合：

6.1 SimAI/README.md

在 Features 或 Limitations 段落加一句：

Pipeline Parallelism (PP) is currently modeled by closed-form formulas (Expose_PP_time and bubble) after per-layer simulation. Fine-grained PP modeling (per-microbatch scheduling with real P2P traffic flowing through ns-3) is on the roadmap (see Issue #283).

6.2 SimAI/docs/Tutorial.md

新增「Pipeline Parallelism Modeling」一节，覆盖：

• 当前公式来源（指向 Layer.cc#L504）
• --pp_overlap 含义、默认值 1 的语义、推荐设置
• 阶段 B/C 的 Roadmap 与跟踪 issue 链接

6.3 SimAI/vidur-alibabacloud/README.md:262

该处已有 simai_simulation 和 simai_analytical 仅模型 TP 通信，不支持 PP 与 EP 的说明，保持一致措辞并交叉引用主 README 新增段落即可。

---

七、缺失信息与假设

为了保持透明，列出我目前的不确定项与默认假设：

序号	缺失/不确定	当前假设
1	本地 aicb / ns-3-alibabacloud / SimCCL 子模块为空目录	假设其行为与 GitHub 上 master 一致；改 AICB 输出格式前需先 git submodule update --init --recursive
2	「细粒度」期望覆盖到哪些调度策略	默认先做 1F1B + Interleaved-1F1B（覆盖 Megatron-LM 主流场景）
3	是否保留 formula 模式作为快速估算	默认双模式共存，--pp_mode 切换，向后兼容
4	阶段 A 的 DP_size 偏大 bug 是否合并到本 issue	默认作为附带 bugfix 一起提，可拆
5	是否需要 PP 通信经过 NVLink（机内）vs RDMA（跨机）分流	默认沿用现有 NCCL Flow Model 的 NVSwitch / NIC 路由逻辑，不引入新拓扑

如果你对上述假设有不同期望，欢迎在 issue 下补充或直接 reply，我们再细化方案。

---

八、详细回答提问中的每个疑问

问 1：现在 PP 还是通过公式估计的吗？

是的。源码位置在 Layer.cc#L504-L507 与 #L734-L737，仅两条闭式公式：

$$ \text{Expose_PP_time} = 2 \cdot \text{vpp} \cdot \text{GA} \cdot \frac{\text{pp_commsize}}{\text{pp_overlap_ratio}} \cdot (1 - \text{pp_overlap_ratio}) $$

$$ \text{Bubble} = \text{step_time} \cdot \frac{\text{PP} - 1}{\text{GA} \cdot \text{vpp}} $$

并且 Sys.cc#L1363, L1376 直接硬编码 PP_size = 1，全代码搜索 ParallelStrategy::PP 零引用，可以确认 PP 通信完全没进入 MockNcclGroup → ns-3 仿真路径。

问 2：之后能支持细粒度的估计吗？

技术上完全可行，且代码已有大量预留：

• enum ParallelStrategy { TP, DP, **PP**, EP, DP_EP, NONE } 已定义
• MockNccl::GroupType { TP, DP, **PP**, EP, DP_EP, NONE } 已定义
• Sys 已有 sim_send / sim_recv API，只差 PP 邻居映射和 P2P SingleFlow 生成

按本回复第四节的三阶段方案推进即可。社区维护方一般倾向于先合阶段 A（修复现有公式法 bug + 文档澄清）回应你的关切，阶段 B/C 进入 Roadmap，欢迎你贡献 PR。

---

九、TL;DR（一句话总结）

是的，目前 PP 是闭式公式估算（Expose_PP_time 与 1F1B bubble 公式），我们建议分三阶段渐进式接入细粒度仿真：A 修 bug → B 把 P2P 注入 ns-3 → C 完整 1F1B/Interleaved 调度器；阶段 A 可作为本 issue 的短期回应立刻合入，向后完全兼容。

如有其他细节需要展开，欢迎在 issue 下继续讨论 :)

EN

Issue #283 Reply (English)

Original Issue: aliyun/SimAI#283 — Will fine-grained PP be supported in the future?
Author: @li-yr
Question: "As the title suggests, PP is currently estimated by formulas. Will fine-grained estimation be supported in the future?"

Thanks for the question. Below is a code-grounded analysis with a Minimum Viable Plan (MVP) answering each of your concerns.

---

Glossary (English / Chinese)

English	中文	Meaning
Pipeline Parallelism (PP)	流水线并行	Split model layers across ranks; chain micro-batches via P2P send/recv
Tensor Parallelism (TP)	张量并行	Slice a single matmul along a dimension across GPUs
Data Parallelism (DP)	数据并行	Different ranks process different mini-batches; AllReduce on gradients
Expert Parallelism (EP)	专家并行	Distribute MoE experts across ranks
Virtual Pipeline Parallelism (VPP)	虚拟流水线并行	Each rank holds multiple non-contiguous chunks (Interleaved-1F1B)
Gradient Accumulation (GA)	梯度累积	Accumulate gradients of several micro-batches before each optimizer step
Micro-batch	微批次	Smallest scheduling unit for PP; mini-batch is split into GA pieces
1F1B (One-Forward-One-Backward)	一前一后调度	Classic steady-state PP schedule alternating fwd / bwd
Bubble	流水线气泡	GPU idle time caused by dependency waits
Closed-form formula	闭式公式	Analytical expression giving the result directly
Point-to-Point (P2P)	点到点通信	Single-sender / single-receiver communication (e.g., send/recv)

---

1. Background Summary

Your observation is correct: PP timing is currently produced by two closed-form formulas as a post-processing step, never entering the simulator's event loop.

Specifically:

• TP / DP / EP collectives are turned into real SingleFlows by MockNcclGroup + the NCCL Flow Model and injected into ns-3-alibabacloud.
• PP communication and PP bubble are added on top of total_time after every layer finishes, by two formulas in Layer.cc.

So "fine-grained PP modeling" — letting PP traffic flow through the same simulator and letting bubbles emerge naturally — is a reasonable and necessary next step.

---

2. Files / Modules Involved

#	Path (GitHub link)	Current Role	Required Change
1	workload/Workload.hh#L76-L83	Holds global PP/vpp/GA/pp_commsize fields	Add stage-layout & schedule fields
2	workload/Workload.cc#L1180-L1268	Parses workload header (pp: vpp: ga: pp_comm:)	Parse pp_schedule: stage_layers:
3	workload/Layer.cc#L504-L507 & #L734-L737	Closed-form post-processing (root cause)	Disable in fine-grained mode
4	system/Sys.cc#L1355-L1397	mock_nccl_grobal_group_init / mock_nccl_comms_init hard-code PP_size = 1	Use real PP_size, build PP comm
5	system/Sys.hh#L49-L56	enum ParallelStrategy { TP, DP, PP, EP, DP_EP, NONE } reserved but unused	Wire into send/recv dispatch
6	system/MockNcclGroup.h#L42-L49 & .cc	GroupType::PP reserved but no neighbor map	Add PP topology + P2P SingleFlow generator
7	system/AstraParamParse.hh#L60 & #L156-L158	pp_overlap_ratio / --pp_overlap	Add --pp_mode --pp_schedule
8	aicb repo	Generates workload files	Emit per-stage layer split & micro-batch info

---

3. Root Cause / Implementation Approach

3.1 Current Formula Mode

Source (Layer.cc#L504-L507):

Tick Expose_PP_time = (2 * vpp * GA *
                       (pp_commsize * GBps / param->net_work_param.pp_overlap_ratio * 1e9)
                       / FREQ);
Expose_PP_time *= (1 - param->net_work_param.pp_overlap_ratio);
// pp bubble time
pre_bubble_time *= static_cast<double>(PP_size - 1) / (GA * vpp);

Mathematically:

$$ \text{Expose_PP_time} ;=; 2 \cdot \text{vpp} \cdot \text{GA} \cdot \frac{\text{pp_commsize}}{\text{pp_overlap_ratio}} \cdot (1 - \text{pp_overlap_ratio}) $$

$$ \text{Bubble} ;=; \text{step_time} \cdot \frac{\text{PP_size} - 1}{\text{GA} \cdot \text{vpp}} $$

Interpretation:

• Line 1: total bidirectional P2P volume × user-supplied "non-overlap ratio" → estimated exposed PP comm time
• Line 2: classical 1F1B bubble formula ($\frac{p-1}{m}$ from the GPipe paper, further reduced by VPP)

Fundamental issue: both are post-hoc multiplications completely disconnected from the simulator's event flow, so they cannot capture:

Real-world Effect	Formula-Mode Behavior
Per-microbatch dependency (recv → fwd → send / recv → bwd → send)	Not modeled
PP P2P contending with TP/DP/EP traffic on shared NIC / link	Not modeled
Bubble differences across schedules (1F1B / Interleaved-1F1B / ZB-H1 / ZB-2P / Chimera)	Single formula for all
Heterogeneous layer split → cross-stage load imbalance	Invisible
Cross-rack / Pod hop count and queueing congestion	Not modeled

3.2 Fine-grained Mode Approach

graph TB
    A[AICB-generated workload<br/>with stage_layers + pp_schedule] --> B[Workload parses PP schedule params]
    B --> C[PipelineScheduler<br/>emits 1F1B event sequence]
    C --> D{Event type}
    D -->|fwd_compute| E[Existing Layer compute path]
    D -->|bwd_compute| E
    D -->|pp_send| F[Sys::sim_send<br/>via NCCL Flow Model]
    D -->|pp_recv| G[Sys::sim_recv<br/>via NCCL Flow Model]
    F --> H[ns-3 real congestion / queueing]
    G --> H
    E --> I[Tick advances<br/>bubble = natural stage wait]
    H --> I
    I --> J[Bubble emerges naturally in Total Time]

Loading

Key insight: in fine mode, bubble = stage idle time emerging naturally (no post-processing needed). PP P2P and TP/DP/EP share the same ns-3 fabric, so contention is captured automatically.

---

4. Minimum Viable Three-Phase Plan (Backward-Compatible)

To honor "minimum diff + no extra deps + match existing style", I propose a three-phase rollout:

Phase A — Fix Formula Accuracy (≤ 50 LoC, immediately mergeable)

Change	Location	Detail
A1	Sys.cc:1363, 1376	int PP_size = 1; → int PP_size = workload->pipeline_model_parallelism; (existing bug: DP_size currently misses dividing by PP, inflating the DP comm group)
A2	Layer.cc:504, 734	Add divide-by-zero guard: fall back to 0 when pp_overlap_ratio == 0
A3	AstraParamParse.hh:60	Document semantics in comment: pp_overlap_ratio = 1 means "100% overlap → exposed=0", note in README

Phase A can land as a standalone PR in direct response to this issue.

Phase B — Inject PP P2P into the Simulator (≈ 200–400 LoC, gated by flag)

Change	Location	Detail
B1	AstraParamParse.hh	Add --pp_mode formula|fine (default formula)
B2	Sys.cc::mock_nccl_grobal_group_init	Build MockNccl::GroupType::PP with real PP_size; record each rank's upstream/downstream neighbor
B3	MockNcclGroup.cc	Add generate_pp_p2p_flow(src, dst, size), reusing SingleFlow
B4	Workload.cc	After each layer, if at a stage boundary AND pp_mode=fine, call sim_send / sim_recv (chunk size pp_commsize / GA)
B5	Layer.cc:504-507, 734-737	Skip closed-form Expose_PP_time post-processing when pp_mode=fine

Phase C — Full PP Scheduler (≈ 500–800 LoC, separate module)

Change	Location	Detail
C1	New astra-sim/workload/PipelineScheduler.{hh,cc}	Abstract base + Schedule1F1B / ScheduleInterleaved (uses vpp) / ScheduleZeroBubble (interface placeholder)
C2	Workload.cc main loop	Driven by scheduler-emitted micro-batch event sequence
C3	aicb repo	Workload header gains pp_schedule: 1f1b|interleaved, stage_layers: …
C4	--pp_schedule CLI flag	Forwarded to Workload

Recommended rollout: land Phase A first to fix the existing formula-mode bug; place Phase B / C on the roadmap.

---

5. Test Prerequisites

Phase	Needs new data?	Detail
A	No	Reuse existing SimAI/example/microAllReduce.txt, workload_analytical.txt; add a TP=2 / PP=2 / DP=2 (8-GPU) regression case to verify the DP_size fix is non-regressive
B	Yes (medium)	Re-generate workload via AICB so the header pp_comm: matches real per-microbatch P2P bytes; reuse inputs/topo/Spectrum-X_*.txt
C	Yes (high)	Need real Megatron-LM training logs (forward / backward / bubble durations) for calibration; at least 7B + 70B reference runs

---

6. README Changes

Yes, minimal change set:

6.1 SimAI/README.md

Add to Features or Limitations:

Pipeline Parallelism (PP) is currently modeled by closed-form formulas (Expose_PP_time and bubble) after per-layer simulation. Fine-grained PP modeling (per-microbatch scheduling with real P2P traffic flowing through ns-3) is on the roadmap (see Issue #283).

6.2 SimAI/docs/Tutorial.md

New section "Pipeline Parallelism Modeling" covering:

• Origin of the current formulas (link to Layer.cc#L504)
• Semantics of --pp_overlap, default 1, recommended values
• Phase B / C roadmap and tracking-issue link

6.3 SimAI/vidur-alibabacloud/README.md:262

Already states simai_simulation and simai_analytical currently only model TP communication and do not support pipeline or expert parallelism; keep wording aligned and cross-reference the new section in the main README.

---

7. Open Questions & Assumptions

For transparency, here are my current uncertainties and default assumptions:

#	Unknown / Uncertain	Default Assumption
1	Local aicb / ns-3-alibabacloud / SimCCL submodules are empty in my workspace	Behavior matches GitHub master; before changing AICB output you must git submodule update --init --recursive
2	How fine-grained should "fine" be?	Start with 1F1B + Interleaved-1F1B (covers mainstream Megatron-LM scenarios)
3	Keep formula mode as a fast estimator?	Yes, dual-mode coexistence via --pp_mode, fully backward-compatible
4	Should the Phase A DP_size bug-fix be folded into this issue?	Default yes (companion bugfix in same PR), can be split
5	Should PP traffic be split across NVLink (intra-host) vs RDMA (inter-host)?	Reuse the existing NCCL Flow Model NVSwitch / NIC routing; do not introduce a new topology

If you have different expectations on any of the above, please reply on the issue and we will refine the plan.

---

8. Detailed Answer to Each Question

Q1: Is PP currently estimated by formulas?

Yes. Source location: Layer.cc#L504-L507 and #L734-L737, only two closed-form formulas:

$$ \text{Expose_PP_time} = 2 \cdot \text{vpp} \cdot \text{GA} \cdot \frac{\text{pp_commsize}}{\text{pp_overlap_ratio}} \cdot (1 - \text{pp_overlap_ratio}) $$

$$ \text{Bubble} = \text{step_time} \cdot \frac{\text{PP} - 1}{\text{GA} \cdot \text{vpp}} $$

In addition, Sys.cc#L1363, L1376 hard-code PP_size = 1 and a full repo search for ParallelStrategy::PP returns zero hits — confirming PP traffic never enters the MockNcclGroup → ns-3 simulation path.

Q2: Will fine-grained estimation be supported in the future?

Technically yes, and the codebase already has many hooks ready:

• enum ParallelStrategy { TP, DP, **PP**, EP, DP_EP, NONE } defined
• MockNccl::GroupType { TP, DP, **PP**, EP, DP_EP, NONE } defined
• Sys::sim_send / sim_recv APIs already exist; we only need PP neighbor mapping + P2P SingleFlow generation

The three-phase plan in §4 is the recommended path. Maintainers typically prefer landing Phase A (formula-mode bug fix + doc clarification) first as the short-term response, with Phase B / C on the roadmap. Community PRs are very welcome.

---

9. TL;DR

Yes, today PP is modeled by closed-form formulas (Expose_PP_time + 1F1B bubble). I recommend a three-phase rollout for fine-grained simulation: A) fix existing bugs → B) inject PP P2P into ns-3 → C) full 1F1B / Interleaved scheduler. Phase A can be merged immediately as the short-term answer to this issue, with full backward compatibility.

Happy to discuss further details under the issue :)

[tianhao909]

[Reply from Agent B] [For reference only]

CN

Issue #283 分析：未来能否支持细粒度 PP 估计？

1. 需求背景总结

Issue 问题是：当前 PP（Pipeline Parallelism，流水线并行）开销主要还是通过公式估计，未来是否能支持更细粒度的估计。

从当前代码看，这个问题是成立的：SimAI Analytical 侧已经能够读取 workload 头部中的 pp、vpp、ga、pp_comm 等字段，但 PP 暴露通信时间仍主要在统计/分析阶段通过一个聚合公式估算，而不是像 TP/DP/EP 通信那样按 workload 中的具体通信事件逐条建模。

当前已有的基础能力包括：

• AICB 已经能生成 Megatron PP 相关的 forward/backward send/recv workload 行为，例如 workload_generator/generate_megatron_workload.py 中会根据 pipeline_model_parallel、pp_rank、num_microbatches 生成 irecv/isend 类型的 PP 通信。
• SimAI workload 格式已经在第一行保留了 pp、vpp、ga、pp_comm 字段。
• SimAI Analytical 当前会读取 pp_comm，并在 Layer.cc 中用公式估计 Expose_PP_comm 与 bubble time。
• Vidur-AlibabaCloud 中已有 send_recv profiling 与 pipeline_parallel_communication_time 统计路径，但 SimAI backend 当前主要细化的是 TP all-reduce，PP 还没有接到 SimAI Analytical/Simulation 的细粒度路径上。

因此建议的 issue 回复可以是：

可以支持，且已有部分前置能力；但当前 SimAI Analytical 中 PP 仍以 pp_comm 聚合公式估算为主。后续可以先以最小改动方式支持基于 send_recv/P2P workload 的细粒度 PP 通信估计，同时保留现有公式作为 fallback。

2. 涉及文件/模块

SimAI

1. docs/Tutorial.md

   • 文档说明 workload 由 AICB 生成。
   • 已说明 workload header 中：
     • model_parallel_NPU_group 表示 TP
     • ep 表示 EP
     • pp 表示 PP
     • vpp 表示虚拟流水线并行

2. example/workload_analytical.txt

   • 示例 workload 第一行已有：
     • pp
     • vpp
     • ga
     • all_gpus
     • pp_comm
   • 说明当前格式已经能承载聚合 PP 信息。

3. astra-sim-alibabacloud/astra-sim/workload/Workload.hh

   • Workload 类已有：
     • pipeline_model_parallelism
     • GA
     • vpp
     • pp_commsize
   • 说明 PP 元信息已经进入 AstraSim/SimAI workload 对象。

4. astra-sim-alibabacloud/astra-sim/workload/Workload.cc

   • initialize_workload() 会解析 workload header：
     • pp:
     • vpp:
     • ga:
     • all_gpus:
     • pp_comm / pp_comm:
   • 并在 pp != 1 但 pp_comm == 0 时打印 workload 格式告警。

5. astra-sim-alibabacloud/astra-sim/workload/Layer.cc

   • Analytical report 中当前使用类似下面的公式计算 PP 暴露通信：
     • Expose_PP_time = 2 * vpp * GA * pp_commsize / pp_overlap_ratio ...
     • 再乘以 (1 - pp_overlap_ratio)
   • 这里是 issue 中“PP 还是通过公式估计”的核心位置。

6. vidur-alibabacloud/vidur/execution_time_predictor/base_execution_time_predictor.py

   • get_execution_time() 中对非最后一个 PP stage 会调用 _get_pipeline_parallel_communication_time(batch)。
   • 但 SimAI backend 当前主要通过 TPTimePredictor 细化 TP 通信，PP 没有同等的 SimAI fine-grained predictor。

7. vidur-alibabacloud/vidur/execution_time_predictor/sklearn_execution_time_predictor.py

   • 已有 _get_pipeline_parallel_communication_time()，使用 send_recv prediction。
   • 可作为细粒度 PP predictor 的现有参考路径。

8. vidur-alibabacloud/vidur/execution_time_predictor/communication_time_predictor.py

   • 当前类名是 TPTimePredictor，核心逻辑是生成单次 ALLREDUCE workload 并调用 SimAI backend。
   • 可以参考这个类新增 PPTimePredictor，生成 SEND_RECV/P2P 或带 pp_comm 的 workload。

9. vidur-alibabacloud/vidur/execution_time_predictor/SimAIWorkload.py

   • SimAIWorkload 已经支持 pp_size 与 pp_comm 字段。
   • 但 WorkItem 当前更偏向 collective 类型，尚未显式表达 PP P2P send/recv 语义。

10. vidur-alibabacloud/vidur/profiling/collectives/*

    • 已有 send_recv profiling：
      • collectives_impl.py
      • collectives_input.py
      • main.py
    • 这可以作为 PP 细粒度估计的 profiling 数据来源或 fallback。

AICB

1. workload_generator/generate_megatron_workload.py

   • with_pipeline_forward_backward() 中已经根据 pp_rank 和 microbatch 生成 irecv/isend。
   • 这是细粒度 PP workload 的主要数据来源。

2. workload_applyer.py

   • 已经有 PP group 生成和 isend/irecv 执行路径。
   • 说明 AICB 侧 PP 语义相对完整。

3. scripts/megatron_workload_with_aiob.sh

   • 已支持 --pipeline_model_parallel / --pp 参数。

ns-3-alibabacloud

ns-3-alibabacloud 更偏底层网络设备、RDMA、QBB、PFC、NVSwitch、队列与拓扑模拟，没有直接的 PP 语义。短期内不建议在 ns-3-alibabacloud 中改 PP 逻辑；PP 应优先在 SimAI/AstraSim workload 与 predictor 层完成映射，再由底层网络模拟执行流量。

3. 根因分析或实现思路

当前根因

当前 PP 估计不够细的主要原因不是缺少 PP 参数，而是 PP 通信被压缩成了 workload header 中的一个聚合字段 pp_comm，然后在 Analytical report 中通过公式估计：

• 公式输入：
  • pp_commsize
  • vpp
  • GA
  • pp_overlap_ratio
  • 带宽相关参数
• 公式输出：
  • Expose_PP_comm
  • bubble time

这种方法适合快速估算，但无法表达：

• 不同 pipeline stage 的 send/recv 差异；
• warmup、steady、cooldown 阶段差异；
• microbatch 级别的 PP 调度；
• virtual pipeline stage 的具体交错；
• PP send/recv 与 TP/EP/DP collective 的重叠关系；
• 跨节点 PP 与节点内 PP 的拓扑差异；
• 不同 message size 下真实 send/recv latency 的非线性。

可行实现思路

建议分两阶段推进，尽量最小改动。

阶段 1：保持公式 fallback，新增细粒度 PP predictor

新增一个和 TPTimePredictor 类似的 PPTimePredictor：

• 输入：
  • batch token 数；
  • hidden size；
  • dtype size；
  • PP size；
  • 当前 pipeline stage；
  • topology/config；
• 计算 PP message size：
  • 训练场景通常近似为 seq_len * micro_batch * hidden_size * dtype_size；
  • 推理场景可以使用 batch._total_num_tokens_rounded * hidden_size * dtype_size 或按 prefill/decode 分开处理；
• 调用：
  • 优先调用 SimAI Analytical/Simulation；
  • 如果失败，fallback 到现有 send_recv profiling 或原公式；
• 缓存：
  • 使用 (hidden_size, num_tokens, dtype_size, pp_size, topology, config) 作为 cache key；
  • 避免每个 batch 重复运行 SimAI。

这样可以不破坏当前 workload 格式，也不影响已有用户。

阶段 2：让 workload 原生表达 PP P2P event

在 workload 格式中加入更明确的 PP P2P 表达，例如：

• 新通信类型：
  • SEND
  • RECV
  • SEND_RECV
  • 或 P2P
• 新 group 类型：
  • PP
• 每条 PP 通信按 microbatch/stage 生成一行。

这样可以把 AICB 中已有的 isend/irecv 信息保留下来，而不是只生成一个 pp_comm 聚合值。

阶段 3：PP 与调度/重叠建模

在精确 PP P2P 通信可用后，再进一步处理：

• warmup/steady/cooldown；
• 1F1B schedule；
• virtual pipeline；
• microbatch accumulation；
• PP bubble；
• PP 与 TP/EP/DP overlap；
• intra-node / inter-node send_recv 差异。

这些属于增强项，不建议放进第一版最小修改。

4. 最小修改方案

建议最小可行方案如下。

方案 A：短期最小方案，适合先回复 issue 并落地初版

1. 在 communication_time_predictor.py 中新增 PPTimePredictor

   • 结构参考 TPTimePredictor。
   • 生成一个最小 PP workload。
   • 先支持 Analytical backend。
   • 失败时返回 -1，由调用方 fallback。

2. 在 base_execution_time_predictor.py 中接入

   • 当前逻辑：
     • 非最后 PP stage 调 _get_pipeline_parallel_communication_time(batch)。
   • 修改思路：
     • 当 backend in ["simai_simulation", "simai_analytical"] 且 num_pipeline_stages > 1 时，优先调用 PPTimePredictor；
     • 如果返回 -1，fallback 到 _get_pipeline_parallel_communication_time(batch)；
     • backend == "aicb" 可暂时继续走现有路径或后续单独接入。

3. 在 SimAIWorkload.py 中补齐 pp_comm

   • 当前 SimAIWorkload 已有 pp_comm 字段。
   • 对 PP predictor 生成 workload 时，将 pp_comm 设置为当前 batch 的 PP message size。
   • 这能让现有 SimAI Analytical 至少基于真实 batch size/token size 估计，而不是完全固定聚合值。

4. 保留 Layer.cc 中现有公式

   • 不建议第一版删除公式。
   • 将其作为 fallback。
   • 这样可以保持兼容已有 workload 与文档。

5. 新增测试

   • 构造一个 pp=2、pp_comm > 0 的小 workload。
   • 验证：
     • parser 能读取 pp_comm；
     • PPTimePredictor 能生成 workload；
     • SimAI Analytical 能返回非负值；
     • 失败时能 fallback，不影响原有行为。

方案 B：中期更完整方案

1. AICB 输出 SimAI workload 时保留 isend/irecv 或转换为 SEND_RECV PP 行。
2. SimAI workload parser 增加 PP P2P 通信类型解析。
3. Layer.cc / workload execution 中按事件级别统计 PP 通信。
4. 文档更新 workload spec。
5. 添加跨节点 PP 与节点内 PP case。

不建议第一版做的事情

• 不建议直接重写 Layer.cc 中所有 PP 统计逻辑。
• 不建议修改 ns-3-alibabacloud 的底层网络模型来表达 PP。
• 不建议引入额外 Python/C++ 依赖。
• 不建议改变现有 workload 格式为不兼容版本。

5. 测试前置条件

不依赖额外数据的测试

以下测试可以用小型人工 workload 完成，不需要额外 profiling 数据：

1. workload header parser 测试

   • 输入包含：
     • pp: 2
     • vpp: 1
     • ga: 1
     • pp_comm: 1024
   • 验证 Workload 中：
     • pipeline_model_parallelism == 2
     • pp_commsize == 1024

2. fallback 测试

   • 模拟 PPTimePredictor 返回 -1。
   • 验证仍然能走原公式或现有 _get_pipeline_parallel_communication_time()。

3. cache key 测试

   • 相同 batch/token/topology 不重复生成 workload 或重复调用 SimAI。

依赖环境/数据的测试

以下测试需要额外环境或数据：

1. SimAI Analytical 端到端测试

   • 需要编译 SimAI_analytical。
   • 需要可用的 workload。
   • 如果启用 bus bandwidth 配置，需要 busbw.yaml。

2. SimAI Simulation / NS-3 测试

   • 需要编译 ns-3 backend。
   • 需要 topology 与 config 文件。
   • 运行时间会更长，不建议作为默认单元测试。

3. AICB 真实 workload 对齐测试

   • 如果要验证 Megatron PP 的 microbatch/stage 级别准确性，需要 AICB 生成的 PP workload。
   • 如果启用 AIOB，需要 GPU、PyTorch、CUDA/NCCL 环境。

缺失信息与假设

缺失信息：

1. issue 期望的“细粒度”具体粒度：

   • microbatch 级？
   • pipeline stage 级？
   • send/recv event 级？
   • 还是只希望 batch-size/token-size aware 的 PP 估计？

2. 目标 backend：

   • SimAI Analytical？
   • SimAI Simulation/NS-3？
   • Vidur-AlibabaCloud inference？
   • 还是训练 workload？

3. PP 通信是否需要考虑 overlap：

   • 是否继续用 pp_overlap_ratio？
   • 是否要由事件调度自然体现 overlap？

4. dtype 与 message size 计算方式：

   • fp16/bf16/fp8？
   • activation tensor 是否包含 sequence parallel 或 tensor parallel 切分？

当前假设：

• 第一版目标是最小改动，不破坏现有 workload；
• 先支持 Analytical 或 Vidur predictor 层的细粒度估计；
• 现有公式继续作为 fallback；
• 不引入新依赖；
• 不改 ns-3-alibabacloud 底层网络模型。

6. README 是否需要修改及建议内容

需要更新 README / docs，但建议在代码功能落地后再更新。最小文档修改建议如下：

建议更新位置

1. docs/Tutorial.md

   • 在 workload 字段说明中补充：
     • pp_comm 的含义；
     • 当前 PP 估计模式；
     • fine-grained PP 开关或限制。

2. vidur-alibabacloud/README.md

   • 如果 PP predictor 接入 Vidur backend，需要更新 backend capability 表。
   • 当前文档中 SimAI Analytical/Simulation 对 TP 支持描述较明确，PP 支持范围需要同步说明。

3. AICB README 或 workload spec

   • 如果新增 PP P2P event 类型，需要更新 workload spec。
   • 如果只是使用 pp_comm 动态估计，则只需要说明生成方式。

建议文案

可以先写成：

PP communication estimation currently supports two modes:

1. Aggregate mode: use pp_comm, vpp, ga, and pp_overlap_ratio to estimate exposed PP communication and pipeline bubbles.
2. Fine-grained mode: when enabled, SimAI estimates PP communication from per-batch/per-stage send-recv message size and falls back to aggregate mode if the fine-grained backend is unavailable.

Fine-grained PP estimation requires valid PP size (pp > 1) and non-zero PP message size. For end-to-end validation with SimAI Simulation, topology and network config files are required.

建议给 issue 的中文回复

可以支持。当前 SimAI Analytical 中 PP 的确主要还是通过 pp_comm + vpp + ga + pp_overlap_ratio 的聚合公式估计，因此对 microbatch、pipeline stage、warmup/steady/cooldown、virtual pipeline 以及跨节点/节点内 send-recv 差异表达不够细。

我们建议按最小改动分阶段支持：

1. 第一阶段先保留现有公式作为 fallback；
2. 新增一个类似 TPTimePredictor 的 PPTimePredictor，根据 batch token 数、hidden size、dtype size、PP size 生成 send-recv/PP workload，并调用 SimAI Analytical 或 Simulation 得到 PP 通信时间；
3. 如果 SimAI backend 不可用或返回异常，则回退到现有公式或现有 send_recv profiling；
4. 第二阶段再考虑让 AICB 生成的 isend/irecv microbatch 级 PP 信息直接进入 SimAI workload，从而支持更细粒度的 pipeline stage / microbatch / virtual pipeline 建模；
5. README 需要补充 PP 估计模式、pp_comm 含义、fine-grained PP 的依赖和限制。

这样可以做到不破坏现有 workload、不引入额外依赖，并逐步从公式估计过渡到事件级 PP 估计。

EN

Issue #283 Analysis: Can SimAI support fine-grained PP estimation in the future?

1. Requirement background

The issue asks whether PP (Pipeline Parallelism) can be estimated in a more fine-grained way, since the current implementation mainly estimates PP with a formula.

This observation is valid. In the current SimAI Analytical path, the workload header already carries PP-related metadata such as pp, vpp, ga, and pp_comm, but the exposed PP communication time is still mostly estimated from an aggregate formula instead of being modeled as concrete per-stage or per-microbatch communication events.

Existing building blocks are already available:

• AICB can generate Megatron PP-related workload behavior. For example, workload_generator/generate_megatron_workload.py generates irecv/isend operations according to pipeline_model_parallel, pp_rank, and num_microbatches.
• The SimAI workload format already contains pp, vpp, ga, and pp_comm in the first line.
• SimAI Analytical currently parses pp_comm and estimates Expose_PP_comm and pipeline bubble time in Layer.cc.
• Vidur-AlibabaCloud already has a send_recv profiling path and a pipeline_parallel_communication_time metric path, but the SimAI backend currently provides finer integration mainly for TP all-reduce, not PP.

A suitable issue response would be:

Yes, fine-grained PP estimation is feasible, and part of the required infrastructure already exists. However, the current SimAI Analytical path mainly uses an aggregate pp_comm formula. A minimal implementation can first add a fine-grained PP predictor based on send/recv or P2P workload while keeping the existing formula as fallback.

2. Related files/modules

SimAI

1. docs/Tutorial.md

   • Documents that workloads are generated by AICB.
   • Explains workload header fields:
     • model_parallel_NPU_group for TP
     • ep for EP
     • pp for PP
     • vpp for virtual pipeline parallelism

2. example/workload_analytical.txt

   • The example workload header already contains:
     • pp
     • vpp
     • ga
     • all_gpus
     • pp_comm
   • This shows that the current format can carry aggregate PP information.

3. astra-sim-alibabacloud/astra-sim/workload/Workload.hh

   • Workload already has:
     • pipeline_model_parallelism
     • GA
     • vpp
     • pp_commsize
   • This means PP metadata is already represented in the AstraSim/SimAI workload object.

4. astra-sim-alibabacloud/astra-sim/workload/Workload.cc

   • initialize_workload() parses:
     • pp:
     • vpp:
     • ga:
     • all_gpus:
     • pp_comm / pp_comm:
   • It also warns when pp != 1 but pp_comm == 0.

5. astra-sim-alibabacloud/astra-sim/workload/Layer.cc

   • The Analytical report currently estimates exposed PP communication with a formula similar to:
     • Expose_PP_time = 2 * vpp * GA * pp_commsize / pp_overlap_ratio ...
     • then multiplied by (1 - pp_overlap_ratio)
   • This is the main location behind the issue's statement that PP is still formula-based.

6. vidur-alibabacloud/vidur/execution_time_predictor/base_execution_time_predictor.py

   • get_execution_time() calls _get_pipeline_parallel_communication_time(batch) for non-last PP stages.
   • However, the SimAI backend currently has a dedicated predictor mainly for TP communication, not PP.

7. vidur-alibabacloud/vidur/execution_time_predictor/sklearn_execution_time_predictor.py

   • Contains _get_pipeline_parallel_communication_time() based on send_recv prediction.
   • This can serve as a reference path for a fine-grained PP predictor.

8. vidur-alibabacloud/vidur/execution_time_predictor/communication_time_predictor.py

   • Currently contains TPTimePredictor, which generates a small ALLREDUCE workload and calls the SimAI backend.
   • A new PPTimePredictor can follow a similar structure.

9. vidur-alibabacloud/vidur/execution_time_predictor/SimAIWorkload.py

   • SimAIWorkload already supports pp_size and pp_comm.
   • WorkItem is still more collective-oriented and does not explicitly model PP P2P send/recv semantics.

10. vidur-alibabacloud/vidur/profiling/collectives/*

    • Already supports send_recv profiling through:
      • collectives_impl.py
      • collectives_input.py
      • main.py
    • This can be used as a profiling source or fallback for PP estimation.

AICB

1. workload_generator/generate_megatron_workload.py

   • with_pipeline_forward_backward() already generates irecv/isend according to pp_rank and microbatch scheduling.
   • This is the primary source for fine-grained PP workload information.

2. workload_applyer.py

   • Already creates PP groups and supports isend/irecv execution paths.
   • This indicates that PP semantics are relatively complete on the AICB side.

3. scripts/megatron_workload_with_aiob.sh

   • Already supports --pipeline_model_parallel / --pp.

ns-3-alibabacloud

ns-3-alibabacloud is mainly a low-level network simulation backend, covering RDMA, QBB, PFC, NVSwitch, queues, topology, and packet-level behavior. It does not directly model PP semantics. Short term, PP should be mapped at the SimAI/AstraSim workload or predictor layer first, and then executed by the lower-level network simulator.

3. Root cause or implementation idea

Root cause

The current limitation is not that PP parameters are missing. The main issue is that PP communication is compressed into a single aggregate workload-header field, pp_comm, and then estimated by a formula in the Analytical report.

The formula depends on:

• pp_commsize
• vpp
• GA
• pp_overlap_ratio
• bandwidth-related parameters

It produces:

• Expose_PP_comm
• pipeline bubble time

This is useful for quick estimation, but it cannot express:

• different send/recv behavior across pipeline stages;
• warmup, steady-state, and cooldown phases;
• microbatch-level PP scheduling;
• virtual pipeline interleaving;
• overlap between PP send/recv and TP/EP/DP collectives;
• intra-node vs inter-node PP topology differences;
• non-linear send/recv latency for different message sizes.

Implementation direction

A staged approach is recommended.

Stage 1: keep formula fallback and add a fine-grained PP predictor

Add a PPTimePredictor similar to TPTimePredictor.

Inputs:

• batch token count;
• hidden size;
• dtype size;
• PP size;
• current pipeline stage;
• topology/config.

Message size estimation:

• For training, a common approximation is:
  • seq_len * micro_batch * hidden_size * dtype_size
• For inference, use:
  • batch._total_num_tokens_rounded * hidden_size * dtype_size
  • or separate prefill/decode handling.

Execution:

• Prefer SimAI Analytical/Simulation backend.
• If unavailable or failed, fall back to existing send_recv profiling or the current formula.

Caching:

• Use a key such as:
  • (hidden_size, num_tokens, dtype_size, pp_size, topology, config)
• Avoid invoking SimAI repeatedly for identical batch shapes.

This keeps the existing workload format compatible.

Stage 2: represent PP P2P events in workload natively

Extend the workload format with explicit PP P2P events, for example:

• new communication type:
  • SEND
  • RECV
  • SEND_RECV
  • or P2P
• new group type:
  • PP
• one workload row per microbatch/stage-level PP communication event.

This allows the isend/irecv information already generated by AICB to be preserved instead of being reduced to a single pp_comm value.

Stage 3: model PP scheduling and overlap

After fine-grained P2P communication is available, further improvements can handle:

• warmup/steady/cooldown;
• 1F1B scheduling;
• virtual pipeline;
• gradient accumulation;
• pipeline bubbles;
• overlap with TP/EP/DP communication;
• intra-node vs inter-node send/recv differences.

These are enhancements and should not be required for the first minimal version.

4. Minimal modification plan

Plan A: short-term minimal version

1. Add PPTimePredictor in communication_time_predictor.py

   • Follow the structure of TPTimePredictor.
   • Generate a minimal PP workload.
   • Support Analytical backend first.
   • Return -1 on failure so the caller can fall back.

2. Wire it into base_execution_time_predictor.py

   • Current logic calls _get_pipeline_parallel_communication_time(batch) for non-last PP stages.
   • Proposed logic:
     • If backend in ["simai_simulation", "simai_analytical"] and num_pipeline_stages > 1, call PPTimePredictor first.
     • If it returns -1, fall back to _get_pipeline_parallel_communication_time(batch).
     • For backend == "aicb", keep the existing path initially or integrate separately later.

3. Use pp_comm in SimAIWorkload.py

   • SimAIWorkload already has pp_comm.
   • For PP predictor-generated workloads, set pp_comm to the current batch's PP message size.
   • This gives the Analytical backend a batch-aware PP size instead of a fully static aggregate value.

4. Keep the existing formula in Layer.cc

   • Do not remove the formula in the first version.
   • Use it as fallback.
   • This preserves compatibility with existing workloads and users.

5. Add tests

   • Create a small pp=2, pp_comm > 0 workload.
   • Verify:
     • parser reads pp_comm;
     • PPTimePredictor can generate workload;
     • SimAI Analytical returns a non-negative value;
     • fallback works when the backend fails.

Plan B: medium-term complete version

1. Preserve AICB-generated isend/irecv events or convert them to SEND_RECV PP rows in SimAI workload.
2. Extend SimAI workload parser to understand PP P2P communication types.
3. Update Layer.cc / workload execution to count PP communication at event level.
4. Update workload spec documentation.
5. Add inter-node and intra-node PP cases.

Not recommended for the first version

• Do not rewrite all PP reporting logic in Layer.cc.
• Do not modify the low-level ns-3-alibabacloud network model just to represent PP.
• Do not introduce new Python/C++ dependencies.
• Do not make an incompatible workload format change.

5. Test prerequisites

Tests not requiring extra data

The following tests can use small handcrafted workloads:

1. Workload header parser test

   • Input header includes:
     • pp: 2
     • vpp: 1
     • ga: 1
     • pp_comm: 1024
   • Verify:
     • pipeline_model_parallelism == 2
     • pp_commsize == 1024

2. Fallback test

   • Simulate PPTimePredictor returning -1.
   • Verify the code falls back to the existing formula or _get_pipeline_parallel_communication_time().

3. Cache key test

   • Same batch/token/topology should not regenerate workload or rerun SimAI.

Tests requiring environment or data

1. SimAI Analytical end-to-end test

   • Requires compiled SimAI_analytical.
   • Requires a valid workload.
   • If bus bandwidth modeling is enabled, requires busbw.yaml.

2. SimAI Simulation / NS-3 test

   • Requires compiled ns-3 backend.
   • Requires topology and config files.
   • Runtime may be long, so it should not be a default unit test.

3. AICB real workload alignment test

   • Requires PP workload generated by AICB.
   • If AIOB is enabled, requires GPU, PyTorch, CUDA, and NCCL.

Missing information and assumptions

Missing information:

1. Expected granularity:

   • microbatch-level?
   • pipeline-stage-level?
   • send/recv-event-level?
   • or only batch-size/token-size-aware PP estimation?

2. Target backend:

   • SimAI Analytical?
   • SimAI Simulation/NS-3?
   • Vidur-AlibabaCloud inference?
   • training workload?

3. Overlap requirement:

   • keep using pp_overlap_ratio?
   • or let the event scheduler naturally model overlap?

4. dtype and message-size formula:

   • fp16/bf16/fp8?
   • should activation tensors account for sequence parallel or tensor parallel sharding?

Assumptions:

• The first version should be minimal and backward-compatible.
• Analytical or Vidur predictor-level PP estimation is the first target.
• The current formula remains as fallback.
• No new dependency is introduced.
• The low-level ns-3 network model is unchanged.

6. README update requirement and suggested content

README/docs should be updated once the feature lands.

Suggested files to update

1. docs/Tutorial.md

   • Document:
     • meaning of pp_comm;
     • current PP estimation modes;
     • fine-grained PP switch or limitations.

2. vidur-alibabacloud/README.md

   • If PP predictor is integrated into the Vidur backend, update the backend capability table.
   • The current TP support statement for SimAI Analytical/Simulation should be updated to clarify PP support scope.

3. AICB README or workload spec

   • If new PP P2P event types are introduced, update the workload spec.
   • If only dynamic pp_comm is used, document how it is generated.

Suggested wording

PP communication estimation currently supports two modes:

1. Aggregate mode: use pp_comm, vpp, ga, and pp_overlap_ratio to estimate exposed PP communication and pipeline bubbles.
2. Fine-grained mode: when enabled, SimAI estimates PP communication from per-batch/per-stage send-recv message size and falls back to aggregate mode if the fine-grained backend is unavailable.

Fine-grained PP estimation requires valid PP size (pp > 1) and non-zero PP message size. For end-to-end validation with SimAI Simulation, topology and network config files are required.

Suggested English issue reply

Yes, this can be supported. The current SimAI Analytical path does mainly estimate PP communication with an aggregate formula based on pp_comm, vpp, ga, and pp_overlap_ratio, so it does not fully capture microbatch-level, stage-level, warmup/steady/cooldown, virtual pipeline, or intra-node/inter-node send-recv differences.

A minimal and backward-compatible plan is:

1. Keep the existing formula as fallback.
2. Add a PPTimePredictor similar to the existing TPTimePredictor. It can generate PP/send-recv workloads from batch token count, hidden size, dtype size, and PP size, then call SimAI Analytical or Simulation to estimate PP communication time.
3. If the SimAI backend is unavailable or fails, fall back to the current formula or existing send_recv profiling path.
4. In a second stage, preserve AICB-generated isend/irecv microbatch-level PP events directly in the SimAI workload format, enabling finer modeling of pipeline stage, microbatch, and virtual pipeline behavior.
5. Update README/docs to describe the PP estimation modes, pp_comm, dependencies, and limitations.

This approach keeps existing workloads compatible, avoids extra dependencies, and allows us to gradually move from formula-based PP estimation to event-level PP estimation.