Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion CHANGELOG.rst
Original file line number Diff line number Diff line change
Expand Up @@ -35,7 +35,8 @@ Changelog

- The ``evaluation`` agent skill can stop looping ("doom loop") reasoning early: ask it to avoid doom loops and it adds vLLM's ``repetition_detection`` request field (vLLM 0.17.0 or later) to every task, ending a request once its output repeats. Use the same setting for baseline and candidate runs.
- Add the ordered DLA compatibility pipeline through the ``modelopt.onnx.graph_surgery.make_dla_compatible``
API and ``make-dla-compatible`` CLI command. Use ``--remove-all-qdq`` to remove Q/DQ pairs of every dtype.
API and ``make-dla-compatible`` CLI command. Use ``--remove-all-qdq`` to remove Q/DQ pairs of every dtype,
or ``--force-fp16-activations`` for FP16 internal computation with FP32 model boundaries.

**Backward Breaking Changes**

Expand Down
9 changes: 9 additions & 0 deletions modelopt/onnx/graph_surgery/__main__.py
Original file line number Diff line number Diff line change
Expand Up @@ -284,6 +284,14 @@ def main():
action="store_true",
help="Remove QDQ pairs for every dtype, matching the C++ remove-all mode",
)
dla_parser.add_argument(
"--force-fp16-activations",
action="store_true",
help=(
"Use FP16 internal activations and weights with FP32 model boundaries; "
"L2 normalization intermediates can overflow or underflow"
),
)

# Analyze subcommand
analyze_parser = subparsers.add_parser(
Expand Down Expand Up @@ -367,6 +375,7 @@ def main():
verbose=not args.quiet,
qdq_quantized_dtypes=args.qdq_quantized_dtypes,
remove_all_qdq=args.remove_all_qdq,
force_fp16_activations=args.force_fp16_activations,
)

elif args.command == "analyze":
Expand Down
Loading
Loading