[calvinbayer]

What Does This Do

Support native MUnit automation exclusion declarations. Wire test declaration metadata into Datadog policy selection and reporting, preserving assigned Test Management properties.

Motivation

Allow individual tests to prevent Datadog quarantine/disabling and exclude Datadog ATR, EFD and Attempt to Fix. Developer selection and framework retries retain their existing semantics.

Additional Notes

Stacked on codex/scalatest-automation-exclusions; review this PR against its base for the framework-specific change. Shared public API and policy are introduced by the first JUnit4 PR. No refactor commit was needed.

Validation: Module spotlessApply and instrumented MUnitTest exclusion regression passed: native tags reach shared policy, protected failure executes once and blocks, assigned properties remain visible, developer-ignored tests retain both tags. Existing MUnit category bridge needs no duplicate runtime implementation.

Snapshot dependency for inherited physical source paths: DataDog/test-environment#604.

test-environment-branch: codex/java-inherited-test-source-snapshots

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	57.39	56.55	$\color{red}{\blacktriangle}$ +0.84	56.55	$\color{red}{\blacktriangle}$ +0.84	185/427
agentEvpProxy	57.57	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	22.76	22.54	$\color{red}{\blacktriangle}$ +0.22	22.54	$\color{red}{\blacktriangle}$ +0.22	123/258
agentless	18.88	19.59	$\color{green}{\blacktriangledown}$ -0.71	19.59	$\color{green}{\blacktriangledown}$ -0.71	118/221
agentlessCodeCoverage	21.09	20.39	$\color{red}{\blacktriangle}$ +0.70	20.39	$\color{red}{\blacktriangle}$ +0.70	118/221
agentlessLineCoverage	27.60	27.52	$\color{red}{\blacktriangle}$ +0.08	27.52	$\color{red}{\blacktriangle}$ +0.08	81/156

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	39.26	37.15	$\color{red}{\blacktriangle}$ +2.11	37.15	$\color{red}{\blacktriangle}$ +2.11	92/225
agentless	39.25	37.15	$\color{red}{\blacktriangle}$ +2.10	37.15	$\color{red}{\blacktriangle}$ +2.10	92/198
agentlessCodeCoverage	47.86	46.30	$\color{red}{\blacktriangle}$ +1.56	45.38	$\color{red}{\blacktriangle}$ +2.48	92/197
agentlessLineCoverage	57.58	56.55	$\color{red}{\blacktriangle}$ +1.03	56.55	$\color{red}{\blacktriangle}$ +1.03	92/197

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[datadog-datadog-prod-us1]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.93% (+0.01%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 55663bc | Docs | Give us feedback!}

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	89.46	89.58	$\color{green}{\blacktriangledown}$ -0.12	89.58	$\color{green}{\blacktriangledown}$ -0.12	92/226
agentless	82.55	82.69	$\color{green}{\blacktriangledown}$ -0.14	82.69	$\color{green}{\blacktriangledown}$ -0.14	92/197
agentlessCodeCoverage	97.59	97.04	$\color{red}{\blacktriangle}$ +0.55	97.04	$\color{red}{\blacktriangle}$ +0.55	92/197
agentlessLineCoverage	114.20	113.88	$\color{red}{\blacktriangle}$ +0.32	113.88	$\color{red}{\blacktriangle}$ +0.32	93/198

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	17.92	10.54	$\color{red}{\blacktriangle}$ +7.38	10.33	$\color{red}{\blacktriangle}$ +7.59	93/225
agentless	12.75	9.92	$\color{red}{\blacktriangle}$ +2.83	9.92	$\color{red}{\blacktriangle}$ +2.83	92/195
agentlessCodeCoverage	21.24	17.38	$\color{red}{\blacktriangle}$ +3.86	17.73	$\color{red}{\blacktriangle}$ +3.51	83/179

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	7.54	9.73	$\color{green}{\blacktriangledown}$ -2.19	9.92	$\color{green}{\blacktriangledown}$ -2.38	92/197

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	12.86	13.67	$\color{green}{\blacktriangledown}$ -0.81	13.40	$\color{green}{\blacktriangledown}$ -0.54	104/239
agentless	11.35	12.37	$\color{green}{\blacktriangledown}$ -1.02	12.37	$\color{green}{\blacktriangledown}$ -1.02	104/209
agentlessCodeCoverage	15.49	15.72	$\color{green}{\blacktriangledown}$ -0.23	15.72	$\color{green}{\blacktriangledown}$ -0.23	104/209
agentlessLineCoverage	17.37	17.38	$\color{green}{\blacktriangledown}$ -0.01	17.73	$\color{green}{\blacktriangledown}$ -0.36	104/209

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	95.48	97.04	$\color{green}{\blacktriangledown}$ -1.56	95.12	$\color{red}{\blacktriangle}$ +0.36	107/241
agentless	92.09	91.39	$\color{red}{\blacktriangle}$ +0.70	91.39	$\color{red}{\blacktriangle}$ +0.70	107/213
agentlessCodeCoverage	99.04	101.00	$\color{green}{\blacktriangledown}$ -1.96	99.00	$\color{red}{\blacktriangle}$ +0.04	107/213
agentlessLineCoverage	101.88	103.04	$\color{green}{\blacktriangledown}$ -1.16	101.00	$\color{red}{\blacktriangle}$ +0.88	117/229

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - okhttp

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	19.51	18.82	$\color{red}{\blacktriangle}$ +0.69	19.59	$\color{green}{\blacktriangledown}$ -0.08	107/240
agentless	18.55	18.45	$\color{red}{\blacktriangle}$ +0.10	19.20	$\color{green}{\blacktriangledown}$ -0.65	106/215
agentlessCodeCoverage	20.48	21.65	$\color{green}{\blacktriangledown}$ -1.17	22.54	$\color{green}{\blacktriangledown}$ -2.06	108/219
agentlessLineCoverage	38.26	37.91	$\color{red}{\blacktriangle}$ +0.35	38.67	$\color{green}{\blacktriangledown}$ -0.41	105/213

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - spring_boot

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	9.33	16.36	$\color{green}{\blacktriangledown}$ -7.03	16.36	$\color{green}{\blacktriangledown}$ -7.03	104/237
agentless	9.75	9.92	$\color{green}{\blacktriangledown}$ -0.17	9.92	$\color{green}{\blacktriangledown}$ -0.17	104/209
agentlessCodeCoverage	13.68	13.94	$\color{green}{\blacktriangledown}$ -0.26	13.67	$\color{red}{\blacktriangle}$ +0.01	104/211
agentlessLineCoverage	21.83	22.99	$\color{green}{\blacktriangledown}$ -1.16	22.54	$\color{green}{\blacktriangledown}$ -0.71	104/209

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-19T22:22:20.031814Z	ba9a78e	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[datadog-datadog-prod-us1]

More details

MUnit’s existing category bridge carries native exclusion tags into shared policy selection and reporting. The added regression and documentation align with that behavior, including preservation of assigned Test Management properties and ignored-test tags.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit ba9a78e}

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-java

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	49.72	14.51	$\color{red}{\blacktriangle}$ +35.21	14.23	$\color{red}{\blacktriangle}$ +35.49	104/237
agentless	9.10	12.37	$\color{green}{\blacktriangledown}$ -3.27	12.87	$\color{green}{\blacktriangledown}$ -3.77	104/211
agentlessCodeCoverage	129.22	84.36	$\color{red}{\blacktriangle}$ +44.86	81.05	$\color{red}{\blacktriangle}$ +48.17	104/211
agentlessLineCoverage	130.99	120.92	$\color{red}{\blacktriangle}$ +10.07	123.36	$\color{red}{\blacktriangle}$ +7.63	104/212

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.73 s	14.69 s	[-0.6%; +1.1%] (no difference)
startup:insecure-bank:tracing:Agent	13.52 s	13.64 s	[-1.7%; -0.2%] (maybe better)
startup:petclinic:appsec:Agent	17.06 s	17.00 s	[-0.6%; +1.3%] (no difference)
startup:petclinic:iast:Agent	16.95 s	16.90 s	[-0.7%; +1.3%] (no difference)
startup:petclinic:profiling:Agent	16.61 s	16.69 s	[-1.5%; +0.5%] (no difference)
startup:petclinic:sca:Agent	17.00 s	16.93 s	[-0.4%; +1.3%] (no difference)
startup:petclinic:tracing:Agent	15.58 s	16.04 s	[-6.9%; +1.2%] (no difference)

Commit: 55663bcd · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[datadog-datadog-prod-us1]

An explicit TestNG transformer override that disables retries is replaced with Datadog’s analyzer, allowing ATR or EFD to rerun tests the user configured to run once.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 6fb7986}

---

Findings that could not be posted inline

dd-java-agent/instrumentation/testng/testng-common/src/main/java/datadog/trace/instrumentation/testng/TestNGUtils.java:78

Preserve transformer overrides that disable retries

When a TestNG 7 IAnnotationTransformer explicitly selects DisabledRetryAnalyzer, hasFrameworkOverride is true, but the native-analyzer check rejects that class and falls through to installing Datadog’s RetryAnalyzer. With ATR or EFD enabled, tests configured to run once can now execute repeatedly. Preserve explicit transformer overrides even when they disable retries, keeping the excluded-test fallback separate.

    if (RetryAnnotationTransformer.hasFrameworkOverride(annotation)) {
      return;
    }
    if (TestExclusions.noAutoRetry(source, Arrays.asList(method.getGroups()))) {

React: 👍 useful · 👎 wrong · 😕 unclear · 👀 right, but not acting on it

What each reaction means

• 👍 Useful: correct, and you're fixing it or will.
• 👎 Wrong: the finding is incorrect, for example the bug can't happen or the code was misread.
• 😕 Unclear: you can't tell what it means or why it matters.
• 👀 Right, not acting: correct, but intended, out of scope for this PR, a known risk, or too minor.

_{🤖 Bits Code Review · Open Bits AI session}

[calvinbayer]

Addressed the finding in this review with the inherited fix d308a2b.

The regression first reproduced the bug: an explicit DisabledRetryAnalyzer override still executed six times with TestNG 7.0 ATR, and five/three times with latest TestNG ATR/EFD. The transformer now preserves every explicit framework override before installing Datadog policy. The full configured suite passes 308 cases with 101 capability skips, plus Muzzle, forbidden-API and module checks. The supported scenarios execute exactly once and emit no retry metadata.

[calvinbayer]

@autotest Review

[datadog-datadog-prod-us1]

More details

MUnit’s existing category bridge forwards declaration tags to shared exclusion policy and reporting, including ignored tests. The added regression coverage and documentation align with that behavior, with no actionable defects reported.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit fcc0cd5}

[calvinbayer]

@autotest Review

[datadog-datadog-prod-us1]

More details

The MUnit declarations align with the existing category bridge and shared exclusion policy. The completed static review found no actionable defect in the regression coverage or documentation.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 6784b3b}

[sarahchen6]

LGTM

[calvinbayer]

@autotest Review

[datadog-datadog-prod-us1]

More details

MUnit’s existing tag bridge carries native exclusion declarations into shared policy and reporting, supporting the documented behavior without additional runtime changes.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 55663bc}