[calvinbayer]

What Does This Do

Support test automation exclusions in JUnit5. Wire test declaration metadata into Datadog policy selection and reporting, preserving assigned Test Management properties.

Motivation

Allow individual tests to prevent Datadog quarantine/disabling and exclude Datadog ATR, EFD and Attempt to Fix. Developer selection and framework retries retain their existing semantics.

Additional Notes

Stacked on codex/junit4-automation-exclusions; review this PR against its base for the framework-specific change. Shared public API and policy are introduced by the first JUnit4 PR. No refactor commit was needed.

Validation: Module spotlessApply and instrumented JUnit5Test.test automation exclusions* passed, proving one execution, blocking failure, both annotation tags, preserved backend assignments, and no attempt-to-fix result.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	57.67	56.55	$\color{red}{\blacktriangle}$ +1.12	56.55	$\color{red}{\blacktriangle}$ +1.12	179/421
agentEvpProxy	57.99	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	90.37	89.58	$\color{red}{\blacktriangle}$ +0.79	89.58	$\color{red}{\blacktriangle}$ +0.79	92/226
agentless	83.28	82.69	$\color{red}{\blacktriangle}$ +0.59	82.69	$\color{red}{\blacktriangle}$ +0.59	92/197
agentlessCodeCoverage	98.44	97.04	$\color{red}{\blacktriangle}$ +1.40	97.04	$\color{red}{\blacktriangle}$ +1.40	90/195
agentlessLineCoverage	114.90	113.88	$\color{red}{\blacktriangle}$ +1.02	113.88	$\color{red}{\blacktriangle}$ +1.02	91/196

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	37.14	37.15	$\color{green}{\blacktriangledown}$ -0.01	37.15	$\color{green}{\blacktriangledown}$ -0.01	91/224
agentless	37.88	37.15	$\color{red}{\blacktriangle}$ +0.73	37.15	$\color{red}{\blacktriangle}$ +0.73	91/197
agentlessCodeCoverage	45.06	46.30	$\color{green}{\blacktriangledown}$ -1.24	45.38	$\color{green}{\blacktriangledown}$ -0.32	90/195
agentlessLineCoverage	55.82	56.55	$\color{green}{\blacktriangledown}$ -0.73	56.55	$\color{green}{\blacktriangledown}$ -0.73	89/194

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	22.38	22.54	$\color{green}{\blacktriangledown}$ -0.16	22.54	$\color{green}{\blacktriangledown}$ -0.16	122/257
agentless	22.40	19.59	$\color{red}{\blacktriangle}$ +2.81	19.59	$\color{red}{\blacktriangle}$ +2.81	117/220
agentlessCodeCoverage	22.58	20.39	$\color{red}{\blacktriangle}$ +2.19	20.39	$\color{red}{\blacktriangle}$ +2.19	118/221
agentlessLineCoverage	30.15	27.52	$\color{red}{\blacktriangle}$ +2.63	27.52	$\color{red}{\blacktriangle}$ +2.63	80/155

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	10.41	9.73	$\color{red}{\blacktriangle}$ +0.68	9.92	$\color{red}{\blacktriangle}$ +0.49	89/194

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	13.19	10.54	$\color{red}{\blacktriangle}$ +2.65	10.13	$\color{red}{\blacktriangle}$ +3.06	92/224
agentless	13.29	9.92	$\color{red}{\blacktriangle}$ +3.37	9.92	$\color{red}{\blacktriangle}$ +3.37	90/193
agentlessCodeCoverage	21.71	17.38	$\color{red}{\blacktriangle}$ +4.33	17.73	$\color{red}{\blacktriangle}$ +3.98	81/177

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[datadog-datadog-prod-us1]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.91% (-0.00%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 2d9d2d2 | Docs | Give us feedback!}

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	13.76	13.67	$\color{red}{\blacktriangle}$ +0.09	13.40	$\color{red}{\blacktriangle}$ +0.36	104/239
agentless	13.00	12.37	$\color{red}{\blacktriangle}$ +0.63	12.37	$\color{red}{\blacktriangle}$ +0.63	104/209
agentlessCodeCoverage	15.66	15.72	$\color{green}{\blacktriangledown}$ -0.06	15.72	$\color{green}{\blacktriangledown}$ -0.06	104/209
agentlessLineCoverage	17.21	17.38	$\color{green}{\blacktriangledown}$ -0.17	17.73	$\color{green}{\blacktriangledown}$ -0.52	101/206

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	98.29	97.04	$\color{red}{\blacktriangle}$ +1.25	95.12	$\color{red}{\blacktriangle}$ +3.17	107/241
agentless	92.83	91.39	$\color{red}{\blacktriangle}$ +1.44	91.39	$\color{red}{\blacktriangle}$ +1.44	107/213
agentlessCodeCoverage	101.91	101.00	$\color{red}{\blacktriangle}$ +0.91	99.00	$\color{red}{\blacktriangle}$ +2.91	107/213
agentlessLineCoverage	103.49	103.04	$\color{red}{\blacktriangle}$ +0.45	101.00	$\color{red}{\blacktriangle}$ +2.49	116/228

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - okhttp

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	17.91	18.82	$\color{green}{\blacktriangledown}$ -0.91	19.59	$\color{green}{\blacktriangledown}$ -1.68	107/240
agentless	19.63	18.45	$\color{red}{\blacktriangle}$ +1.18	19.20	$\color{red}{\blacktriangle}$ +0.43	106/215
agentlessCodeCoverage	20.68	21.65	$\color{green}{\blacktriangledown}$ -0.97	22.54	$\color{green}{\blacktriangledown}$ -1.86	108/219
agentlessLineCoverage	37.75	37.91	$\color{green}{\blacktriangledown}$ -0.16	38.67	$\color{green}{\blacktriangledown}$ -0.92	102/210

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - spring_boot

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	16.54	16.36	$\color{red}{\blacktriangle}$ +0.18	16.36	$\color{red}{\blacktriangle}$ +0.18	104/237
agentless	10.03	9.92	$\color{red}{\blacktriangle}$ +0.11	9.92	$\color{red}{\blacktriangle}$ +0.11	104/209
agentlessCodeCoverage	14.35	13.94	$\color{red}{\blacktriangle}$ +0.41	13.67	$\color{red}{\blacktriangle}$ +0.68	104/211
agentlessLineCoverage	22.65	22.99	$\color{green}{\blacktriangledown}$ -0.34	22.54	$\color{red}{\blacktriangle}$ +0.11	102/207

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[dd-octo-sts]

🟡 Java Benchmark SLOs — Performance SLO warning (near threshold)

Suite	Status
Startup	🟡 warning

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.93 s	13.83 s	[+0.1%; +1.4%] (maybe worse)
startup:insecure-bank:tracing:Agent	12.91 s	12.91 s	[-1.0%; +1.0%] (no difference)
startup:petclinic:appsec:Agent	17.72 s	17.41 s	[+1.0%; +2.6%] (maybe worse)
startup:petclinic:iast:Agent	17.51 s	17.48 s	[-0.7%; +1.0%] (no difference)
startup:petclinic:profiling:Agent	17.53 s	17.34 s	[+0.1%; +2.1%] (maybe worse)
startup:petclinic:sca:Agent	17.75 s	17.60 s	[-0.1%; +1.7%] (no difference)
startup:petclinic:tracing:Agent	16.57 s	16.29 s	[-2.7%; +6.2%] (no difference)

Commit: 2d9d2d23 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-java

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	26.93	14.51	$\color{red}{\blacktriangle}$ +12.42	14.23	$\color{red}{\blacktriangle}$ +12.70	104/237
agentless	-2.63	12.37	$\color{green}{\blacktriangledown}$ -15.00	12.87	$\color{green}{\blacktriangledown}$ -15.50	104/211
agentlessCodeCoverage	97.30	84.36	$\color{red}{\blacktriangle}$ +12.94	81.05	$\color{red}{\blacktriangle}$ +16.25	104/211
agentlessLineCoverage	135.09	120.92	$\color{red}{\blacktriangle}$ +14.17	123.36	$\color{red}{\blacktriangle}$ +11.73	103/211

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-19T22:13:36.619184Z	0b80b18	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[datadog-datadog-prod-us1]

More details

JUnit 5 skip selection receives declaration metadata consistently with the existing retry-policy and reporting paths. The completed static review identified no actionable regression against the supplied base.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 0b80b18}

[datadog-datadog-prod-us1]

The new exclusion resolver lacks coverage in internal-api’s own JaCoCo report, causing the required Java 8 coverage gate to fail despite tests in dependent modules.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 9e1ec7c}

---

Findings that could not be posted inline

internal-api/src/main/java/datadog/trace/api/civisibility/config/TestExclusions.java:12

Add owning-module coverage for the exclusion resolver

Java 8 CI runs baseTest with -PcheckCoverage, enforcing internal-api’s per-class minimums of 70% branch and 80% instruction coverage. This new helper has no module-local tests or coverage exclusion; tests added in other modules do not contribute to its report. The required coverage verification therefore fails. Add internal-api tests for the resolver and cover or explicitly exclude the newly executable TestEventsHandler.skipReason default method.

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means

• 👍 Useful: correct, and you're fixing it or will.
• 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
• 😕 Unclear: you can't tell what it means or why it matters.
• 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

_{🤖 Bits Code Review · Open Bits AI session}

[calvinbayer]

Addressed the coverage finding in this AutoTest review with 8028762, which this PR inherits from its JUnit4 parent.

The owning-module JaCoCo gate failed before the test additions: TestExclusions and the new default handler callbacks had zero coverage. Four durable behavior cases now cover independent annotations, tag resolution, empty/unknown metadata, and compatibility delegation through the legacy callbacks. On current master, the complete internal-api suite passes 5,256 tests and the coverage gate passes with 100% instruction and branch coverage for TestExclusions and 100% instruction coverage for the added TestEventsHandler defaults. No coverage thresholds were lowered.

[calvinbayer]

@autotest Review

[datadog-datadog-prod-us1]

More details

The JUnit 5 skip decision now receives declaration metadata and tags consistently with the existing retry-policy and reporting paths. The completed static review identified no actionable regression.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 73c9c70}

[calvinbayer]

@autotest Review

[datadog-datadog-prod-us1]

More details

Method annotations and JUnit tags reach the exclusion-aware skip policy, consistent with the metadata already used for retries and reporting. Native skips and developer test selection retain their existing semantics.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 2d9d2d2}