OPS-157454: Upload test coverage to Codacy from CI - #52
Conversation
Adds a coverage job that runs the test project as a Microsoft.Testing.Platform executable with Microsoft.Testing.Extensions.CodeCoverage and uploads the Cobertura report to Codacy, on the self-hosted pdl-public runners. Excluded tests: none - every test is offline (847 tests). Local line coverage (clean Linux container, no user secrets): 85.4% Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Up to standards ✅🟢 Issues
|
| Metric | Results |
|---|---|
| Coverage variation | Report missing for 2ede4f01 |
| Diff coverage | ✅ ∅ diff coverage |
Coverage variation details
Coverable lines Covered lines Coverage Common ancestor commit (2ede4f0) Report Missing Report Missing Report Missing Head commit (d88c935) 3022 2542 84.12% Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch:
<coverage of head commit> - <coverage of common ancestor commit>
Diff coverage details
Coverable lines Covered lines Diff coverage Pull request (#52) 0 0 ∅ (not applicable) Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified:
<covered lines added or modified>/<coverable lines added or modified> * 100%
1 Codacy didn't receive coverage data for the commit, or there was an error processing the received data. Check your integration for errors and validate that your coverage setup is correct.
AI Reviewer: first review requested successfully. AI can make mistakes. Always validate suggestions.
TIP This summary will be updated as you push new changes.
There was a problem hiding this comment.
Pull Request Overview
The PR successfully implements the framework for coverage reporting but contains critical pathing errors and a missing dependency file that will prevent the CI job from completing as intended. While Codacy indicates the PR is up to standards, the 'coverage' job will likely fail in its current state.
The primary issues to address are the mismatch between the generated Cobertura report location and the paths specified in the upload steps, and the inclusion of the 'coverage.config' file referenced in the workflow. There is also an opportunity to improve CI performance by removing redundant test execution.
About this PR
- The workflow is configured to use 'PanoramicData.OData.Client.Test/coverage.config' for exclusions, but this file is missing from the repository. This contradicts the acceptance criteria regarding code exclusions and will lead to job failure.
Test suggestions
- Verify the CI job executes the test binary with coverage enabled and generates the Cobertura XML file.
- Verify that coverage exclusions defined in 'coverage.config' are successfully applied during the test run.
- Verify the coverage report is successfully uploaded to Codacy.
- Verify the coverage report is archived as a build artifact with a 7-day retention period.
Prompt proposal for missing tests
Consider implementing these tests if applicable:
1. Verify that coverage exclusions defined in 'coverage.config' are successfully applied during the test run.
TIP Improve review quality by adding custom instructions
TIP How was this review? Give us feedback
| run: >- | ||
| ./PanoramicData.OData.Client.Test/bin/Debug/net10.0/PanoramicData.OData.Client.Test | ||
| --coverage | ||
| --coverage-settings PanoramicData.OData.Client.Test/coverage.config |
There was a problem hiding this comment.
🔴 HIGH RISK
The file 'PanoramicData.OData.Client.Test/coverage.config' is missing from the PR. Please include this file to ensure the coverage run can complete successfully with the intended exclusions.
| run: >- | ||
| ./PanoramicData.OData.Client.Test/bin/Debug/net10.0/PanoramicData.OData.Client.Test | ||
| --coverage | ||
| --coverage-settings PanoramicData.OData.Client.Test/coverage.config | ||
| --coverage-output-format cobertura | ||
| --coverage-output coverage.cobertura.xml | ||
|
|
||
| # continue-on-error: a Codacy outage must not turn a passing test run red. The | ||
| # trade-off is that an expired or missing project token fails silently, so | ||
| # check the step log for "Coverage received successfully". | ||
| - name: Upload coverage to Codacy | ||
| continue-on-error: true | ||
| uses: codacy/codacy-coverage-reporter-action@v1 | ||
| with: | ||
| project-token: ${{ secrets.CODACY_PROJECT_TOKEN }} | ||
| coverage-reports: PanoramicData.OData.Client.Test/bin/Debug/net10.0/TestResults/coverage.cobertura.xml | ||
|
|
||
| - name: Upload coverage artifact | ||
| if: always() | ||
| continue-on-error: true | ||
| uses: actions/upload-artifact@v7 | ||
| with: | ||
| name: coverage | ||
| path: PanoramicData.OData.Client.Test/bin/Debug/net10.0/TestResults/coverage.cobertura.xml |
There was a problem hiding this comment.
🔴 HIGH RISK
The coverage report path in the upload steps is incorrect. The test runner generates the report based on the working directory (root) or the --coverage-output flag (line 86), but the upload steps (lines 96, 104) look in the project's 'bin/Debug' folder. To fix this, explicitly set the results directory and use a consistent path:
- Add
--results-directory TestResultsto the test command. - Update the upload steps to use 'TestResults/coverage.cobertura.xml'.
Adds a
coveragejob to CI that collects test coverage and uploads it to Codacy.UseMicrosoftTestingPlatformRunner) withMicrosoft.Testing.Extensions.CodeCoverage, writing a Cobertura report.coverage.configports the generated-code exclusions used across the estate, so figures stay comparable.codacy/codacy-coverage-reporter-actionusing theCODACY_PROJECT_TOKENsecret (continue-on-error, so a Codacy outage cannot turn a passing run red), and kept as a 7-day artifact.pdl-publicrunners, not a GitHub-hosted runner.Excluded tests: none - every test is offline (847 tests).
CI runs only tests that need no credentials. Tests are selected by what passes in a clean Linux container with no user secrets, exactly as CI sees it.
Local line coverage: 85.4%
Jira: https://jira.panoramicdata.com/browse/OPS-157454
🤖 Generated with Claude Code