Repository navigation

[OMNIML-6227] Convert GGML checkpoints to INT8 compressed-tensors for vLLM - #2903

Merged

cjluo-nv merged 7 commits into

mainfrom

chenjiel/ggml-int8-export

Oct 21, 2026

Merged

[OMNIML-6227] Convert GGML checkpoints to INT8 compressed-tensors for vLLM#2903
cjluo-nv merged 7 commits into
mainfrom
chenjiel/ggml-int8-export

Commits on Oct 20, 2026
- Export GGML-quantized layers as INT8 compressed-tensors for vLLM
  
  cjluo-nv
  and
  claude
  committed
  3a04c02
- Export GGML INT8 from the in-memory model, beside the vLLM fake-quant export
  
  cjluo-nv
  and
  claude
  committed
  2e7b207
Commits on Oct 21, 2026
- Convert GGML exports to INT8 with a script beside hf_ptq.py, not an export flag
  
  cjluo-nv
  and
  claude
  committed
  94a42be
- Fold the GGML-to-INT8 converter into convert_ggml_to_int8.py
  
  cjluo-nv
  and
  claude
  committed
  0484374
- Keep nested sidecars and clean up after a failed GGML-to-INT8 conversion
  
  cjluo-nv
  and
  claude
  committed
  2244ea4
- Move the GGML-to-INT8 tests under tests/examples and refuse unconvertible payloads
  
  cjluo-nv
  and
  claude
  committed
  7607678
- Run the converter with sys.executable in the vLLM example test
  
  cjluo-nv
  and
  claude
  committed
  d1dad23