Repository navigation

Megatron-Bridge distillation: packed chat-SFT data prep and per-document attention - #2912

Merged

kevalmorabia97 merged 6 commits into

mainfrom

kmorabia/sft-packing-upstream

Oct 23, 2026

Merged

Megatron-Bridge distillation: packed chat-SFT data prep and per-document attention#2912
kevalmorabia97 merged 6 commits into
mainfrom
kmorabia/sft-packing-upstream

Commits on Oct 21, 2026
- Prepare packed chat-SFT data and add per-document attention to Megatron-Bridge distillation
  
  kevalmorabia97
  and
  claude
  committed
  05046ac
- Use the student EOS as the document-boundary EOD; fold its test into the HF-export one
  
  kevalmorabia97
  and
  claude
  committed
  24155ea
Commits on Oct 22, 2026
- Test the document-boundary cu_seqlens; fix the mock-data segment comment
  
  kevalmorabia97
  and
  claude
  committed
  26d20f7
- docs: note that CP padding can push a packed pair past pack_max_tokens
  
  kevalmorabia97
  and
  claude
  committed
  2d6c33a
Commits on Oct 23, 2026
- Merge remote-tracking branch 'origin/main' into kmorabia/sft-packing-upstream
  
  kevalmorabia97
  and
  claude
  committed
  a60d5df
- Keep assistant-less rows for loss_mode full; raise chat-tokenization setup errors
  
  kevalmorabia97
  and
  claude
  committed
  17964ed