Repository navigation
Add an overhead benchmark workflow for AD vs double builds - #55
Merged
Merged
Conversation
Closed
Contributor
Author
|
CI against QuantLib master plus lballabio/QuantLib#3068 (which fixes master's test suite with AD types): all jobs pass. https://github.com/auto-differentiation/QuantLibAAD/actions/runs/43262133585 The red checks on this PR come from QuantLib master's test suite not compiling with AD until lballabio/QuantLib#3068 is merged. |
auto-differentiation-dev
deleted the
feature/overhead-benchmarking-v2
branch
November 12, 2026 18:08
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a workflow that measures what XAD's active type costs QuantLib when no derivatives are computed. It builds QuantLib twice on the same runner with QuantLibAAD's build defaults, once with
Real = xad::AReal<double>and once withReal = double, times every test-suite case and the QuantLib benchmark in interleaved, repeated runs, and reports AD/double time ratios per test. Ratios rather than absolute times, because hosted runners are noisy. Builds on #54, whose defaults it measures.Changes
.github/workflows/overhead.yaml: manual run with inputs for the XAD ref (defaultv2.2.0), the QuantLib ref (default a recent master commit that builds with AD; master itself sometimes needs an upstream fix first) and the number of repeats (default 3), and arepository_dispatchhook of typexad-release(client_payload.xad_ref, optionallyql_ref) that other repositories can use. Jobs for Linux GCC, Linux Clang and Windows MSVC.tools/benchmark/run_benchmarks.py: runs the test suite (one Boost.Test JUNIT log per run) and the benchmark of both builds, alternating which build goes first. It checks that the two build dirs are a double and an AD build made with the same compiler, and that they differ in nothing else, and records the build options with the results. A failing test or a benchmark run without its timing table fails the job, but the summary is still written.tools/benchmark/aggregate.py: per-test medians and ratios toresults.csv, and a job summary with the total run time ratio, the median and geometric mean of the per-test ratios, the 20 tests with the highest overhead and the number of runs used. Failed tests, benchmark tests that threw and runs that crashed or can't be read are left out; a source without any usable run fails the job.tools/benchmark/README.md: how to run it on GitHub and locally, and what the numbers mean..github/workflows/ci.yaml: a small job that runs the scripts' unit tests on every push and pull request.Testing
python3 -m pytest tools/benchmark/tests(log parsing, broken or empty logs, interleaving, medians, ratios, geometric mean, worst tests, CSV and summary output, build and argument checks).Closes #33