Repository navigation

Tool: evaluate layer-wise numerical-error propagation - #525

Open

jlamypoirier wants to merge 45 commits into

mainfrom

jlp_evaluate_precision

Commits on May 27, 2026
- Add tool to evaluate layer-wise numerical-error propagation
  
  jlamypoirier
  and
  claude
  committed
  02fd39c
- Collapse to a single config; require a checkpoint
  
  jlamypoirier
  and
  claude
  committed
  4dd6c14
- Expose `model:` alongside `pretrained:` in the tool config
  
  jlamypoirier
  and
  claude
  committed
  5ebea33
- Inherit PretrainedGPTModelConfig; use Config update mechanism
  
  jlamypoirier
  and
  claude
  committed
  4c444d8
- Expand HF metadata allowlist for newer transformers configs
  
  jlamypoirier
  and
  claude
  committed
  35206a6
- Reshape console table for readability
  
  jlamypoirier
  and
  claude
  committed
  bde1efa
- Merge tensor+kind, fix decimal precision in console table
  
  jlamypoirier
  and
  claude
  committed
  8099b51
- Switch back to fixed-decimal formatting in the table
  
  jlamypoirier
  and
  claude
  committed
  dbd7702
- Wipe per-variant experiment dir before each run
  
  jlamypoirier
  and
  claude
  committed
  152ffc3
Commits on May 28, 2026
- Support pre-generated memmap dataset; misc table-format polish
  
  jlamypoirier
  and
  claude
  committed
  7e98500
- Print per-variant summary at the end of the run
  
  jlamypoirier
  and
  claude
  committed
  173ae0d
- Reshape end-of-run summary: variants × aggregations, relative only
  
  jlamypoirier
  and
  claude
  committed
  005fd62
- Clarify intermediate aggregation in summary header
  
  jlamypoirier
  and
  claude
  committed
  c594658
- Split summary across fw/bw rows; one extra precision digit
  
  jlamypoirier
  and
  claude
  committed
  3159f73
- Two-row column header in summary; chronological column order
  
  jlamypoirier
  and
  claude
  committed
  6ef153e
- Add fp32_lm_head flag for vLLM precision parity
  
  jlamypoirier
  and
  claude
  committed
  7327932
- Extract layer-name labels for summary first/last columns
  
  jlamypoirier
  and
  claude
  committed
  76335df
- Add `debug_hidden_states_log` to capture named tensors via output_hidden_states
  
  jlamypoirier
  and
  claude
  committed
  8122946
- Capture logit gradients; expose them in the summary
  
  jlamypoirier
  and
  claude
  committed
  4633bfd
- Place logits after head in bw summary; widen format for sub-percent values
  
  jlamypoirier
  and
  claude
  committed
  9ca1711
- Pick per-column decimals to guarantee ≥2 sig figs
  
  jlamypoirier
  and
  claude
  committed
  f2655f3
- Tighten summary table spacing
  
  jlamypoirier
  and
  claude
  committed
  7f8ef96
- Support HF Hub model ids in pretrained.path
  
  jlamypoirier
  and
  claude
  committed
  08b1637
- Add example precision-evaluation configs
  
  jlamypoirier
  and
  claude
  committed
  77eae22
- Drop bf16_no_fp32_gradients variant from example configs
  
  jlamypoirier
  and
  claude
  committed
  efa95b1
- Add weight gradients to per-variant report tables
  
  jlamypoirier
  and
  claude
  committed
  46bc5b8
- Separate fw/bw/grad rows in per-variant tables
  
  jlamypoirier
  and
  claude
  committed
  bef2f0d
- Split summary into three tables (fw, bw, grad)
  
  jlamypoirier
  and
  claude
  committed
  4fecad4
- Split grad summary by parameter category
  
  jlamypoirier
  and
  claude
  committed
  4f47dc0
- Per-tensor sample-density overrides in TensorLogsConfig
  
  jlamypoirier
  and
  claude
  committed
  5198c25
Commits on May 29, 2026
- Chosen-logprob loss, per-variant grad-scale auto-calibration, fp16 variants
  
  jlamypoirier
  and
  claude
  committed
  312343e
Commits on Jun 1, 2026
- Lean fixed-input runner + DeepSpeed-side precision comparison
  
  jlamypoirier
  and
  claude
  committed
  497c76c
- Support random-init (model_weights=False) in both precision tools
  
  jlamypoirier
  and
  claude
  committed
  cecf7ae
- vLLM within-engine precision tool
  
  jlamypoirier
  and
  claude
  committed
  a6d9314
- Auto-bind vLLM fp32 head on tied-embedding models
  
  jlamypoirier
  and
  claude
  committed
  26cd8ab
Commits on Jun 2, 2026
- Cross-engine log-prob comparison tool + per-token log π persistence
  
  jlamypoirier
  and
  claude
  committed
  fc6072c
- Cross-engine: all-pairwise mismatched group + fp16 precision
  
  jlamypoirier
  and
  claude
  committed
  767e7eb
- Cross-engine: full 2x2 head matrix per pair, corrected mismatch direction
  
  jlamypoirier
  and
  claude
  committed
  2828c35
- Add forward-only inference mode to the precision tool
  
  jlamypoirier
  and
  claude
  committed
  ca0c7b2
- Add forward-only mode to the DeepSpeed precision tool
  
  jlamypoirier
  and
  claude
  committed
  f2b9d21
- Add Qwen2.5-7B precision-evaluation config (forward-only)
  
  jlamypoirier
  and
  claude
  committed
  32c9545
Commits on Jun 3, 2026
- Multi-sequence (per-sequence/GSPO) cross-engine log-prob mode
  
  jlamypoirier
  and
  claude
  committed
  f8a3d20
Commits on Jun 4, 2026
- Multi-sequence forward+backward layer-wise mode + network-FS-safe rmtree
  
  jlamypoirier
  and
  claude
  committed
  7a203aa
Commits on Jun 6, 2026
- Merge origin/main into jlp_evaluate_precision
  jlamypoirier
  committed
  35a913d
Commits on Jun 8, 2026
- Batch-size scaling experiment + read-only memmap and TF32 matmul fixes
  
  jlamypoirier
  and
  claude
  committed
  116dc06

Commits

Commits on May 27, 2026

Commits on May 28, 2026

Commits on May 29, 2026

Commits on Jun 1, 2026

Commits on Jun 2, 2026

Commits on Jun 3, 2026

Commits on Jun 4, 2026

Commits on Jun 6, 2026

Commits on Jun 8, 2026