Skip to content

[CI] Add nightly ELD code coverage workflow with regression detection - #1405

Open
Saharsh Burra (bsaharsh) wants to merge 1 commit into
qualcomm:mainfrom
bsaharsh:add-coverage-regression-detection
Open

Saharsh Burra (bsaharsh) wants to merge 1 commit into
qualcomm:mainfrom
bsaharsh:add-coverage-regression-detection

Conversation

@bsaharsh

Copy link
Copy Markdown
Contributor

Added a workflow which runs nightly and computes the code coverage, and when ran at first creates a baseline text file on orphan branch called code-coverage, and from the next time, it checks with the previous coverage numbers and if any metric goes down, it fails, and if no regression is detected then we push the html report and update the baseline text file.

@bsaharsh
Saharsh Burra (bsaharsh) force-pushed the add-coverage-regression-detection branch from 7ab5f2f to 5cb95b0 Compare July 7, 2026 04:10
git config user.name 'github-actions'
git config user.email 'github-actions@github.com'

git checkout --orphan coverage-update

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we have this branch as code-coverage-reports ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated the name

jobs:
coverage:
if: github.repository == 'qualcomm/eld'
runs-on: self-hosted

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shankar Easwaran (@quic-seaswara) is it okay to run this workflow on self-hosted? self-hosted here will make adding this workflow easier because we would not need to install/setup the cross-toolchains and qemu.

repository: ${{ github.repository }}
ref: ${{ github.ref_name }}
path: llvm-project/llvm/tools/eld
fetch-depth: 0

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can likely remove fetch-depth: 0 from here.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated

../llvm-project/llvm \
-DCMAKE_BUILD_TYPE=Debug \
-DCMAKE_DISABLE_PRECOMPILE_HEADERS=ON \
-DLLVM_ENABLE_PROJECTS="llvm;clang;clang-tools-extra" \

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to use the clang and the other tools from the nightly eld toolchain and only build eld with code-coverage enabled?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Updated, but need to test it as fork won't be having the self hosted, may be we can use the workflow dispatch.

@bsaharsh
Saharsh Burra (bsaharsh) force-pushed the add-coverage-regression-detection branch from 5cb95b0 to 0149db3 Compare July 8, 2026 03:50
Comment thread .github/workflows/nightly-eld-code-coverage.yml Fixed
Comment thread .github/workflows/nightly-eld-code-coverage.yml Fixed

@Steven6798 Steven Ramirez Rosa (Steven6798) left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Make sure your code doesn't have any Semgrep Findings if it is preventable. Most of these issues will be pointed out by the github-advanced-security bot. Look at other workflows to see how tags for GitHub actions are written.

@bsaharsh
Saharsh Burra (bsaharsh) force-pushed the add-coverage-regression-detection branch 2 times, most recently from 531af9e to 60cec40 Compare July 10, 2026 06:41
@parth-07
Parth (parth-07) force-pushed the add-coverage-regression-detection branch 3 times, most recently from aa5537f to eb451ab Compare July 14, 2026 15:07
@bsaharsh
Saharsh Burra (bsaharsh) force-pushed the add-coverage-regression-detection branch from eb451ab to 10b8ddc Compare July 14, 2026 16:33
@parth-07
Parth (parth-07) force-pushed the add-coverage-regression-detection branch from 49d7a22 to 4cd34c6 Compare July 15, 2026 11:01
@bsaharsh
Saharsh Burra (bsaharsh) force-pushed the add-coverage-regression-detection branch from 4cd34c6 to 8fcbdc7 Compare July 15, 2026 11:32
@parth-07
Parth (parth-07) force-pushed the add-coverage-regression-detection branch from 85f166b to 445434e Compare July 15, 2026 15:33
@bsaharsh
Saharsh Burra (bsaharsh) force-pushed the add-coverage-regression-detection branch 2 times, most recently from f7bb343 to 7e09cbc Compare July 16, 2026 05:33
@parth-07
Parth (parth-07) force-pushed the add-coverage-regression-detection branch from 7e09cbc to 15b56b4 Compare July 17, 2026 09:26
@parth-07
Parth (parth-07) self-requested a review July 17, 2026 09:32
Add nightly-eld-code-coverage.yml which runs daily after the nightly build.
The workflow:
- Builds LLVM+ELD with source-based coverage instrumentation (ELD_COVERAGE=ON)
- Runs check-eld under LLVM_PROFILE_FILE to collect per-process .profraw files
- Merges profiles and generates an HTML report with llvm-profdata / llvm-cov
- Compares Line/Function/Region/Branch coverage against a baseline stored on
  the code-coverage-reports branch and fails if any metric regresses
- Records the baseline on the first run when none exists
- Publishes the HTML report and updated baseline to the code-coverage-reports
  branch

Also updates CMakeLists.txt: ELD_COVERAGE now sets the llvm-cov source-based
instrumentation flags (-fprofile-instr-generate -fcoverage-mapping).

Signed-off-by: bsaharsh <bsaharsh@qti.qualcomm.com>
@bsaharsh
Saharsh Burra (bsaharsh) force-pushed the add-coverage-regression-detection branch from 15b56b4 to 96e0a2c Compare July 22, 2026 11:23

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants