[calvinbayer]

What Does This Do

Support test automation exclusions in Spock. Wire test declaration metadata into Datadog policy selection and reporting, preserving assigned Test Management properties.

Motivation

Allow individual tests to prevent Datadog quarantine/disabling and exclude Datadog ATR, EFD and Attempt to Fix. Developer selection and framework retries retain their existing semantics.

Additional Notes

Stacked on codex/testng-automation-exclusions; review this PR against its base for the framework-specific change. Shared public API and policy are introduced by the first JUnit4 PR. No refactor commit was needed.

Validation: Module spotlessApply and instrumented SpockTest.test automation exclusions* passed, covering protected execution through spec-level skip aggregation, one execution, blocking failure, annotation tags and retained backend assignments.

Snapshot dependency for inherited physical source paths: DataDog/test-environment#604.

test-environment-branch: codex/java-inherited-test-source-snapshots

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sbt-scalatest

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	56.40	56.55	$\color{green}{\blacktriangledown}$ -0.15	56.55	$\color{green}{\blacktriangledown}$ -0.15	181/423
agentEvpProxy	56.46	n/a	n/a	n/a	n/a	-

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - netflix-zuul

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	91.06	89.58	$\color{red}{\blacktriangle}$ +1.48	89.58	$\color{red}{\blacktriangle}$ +1.48	92/226
agentless	83.54	82.69	$\color{red}{\blacktriangle}$ +0.85	82.69	$\color{red}{\blacktriangle}$ +0.85	92/197
agentlessCodeCoverage	97.72	97.04	$\color{red}{\blacktriangle}$ +0.68	97.04	$\color{red}{\blacktriangle}$ +0.68	91/196
agentlessLineCoverage	114.00	113.88	$\color{red}{\blacktriangle}$ +0.12	113.88	$\color{red}{\blacktriangle}$ +0.12	91/196

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - nebula-release-plugin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	37.54	37.15	$\color{red}{\blacktriangle}$ +0.39	37.15	$\color{red}{\blacktriangle}$ +0.39	92/225
agentless	37.26	37.15	$\color{red}{\blacktriangle}$ +0.11	37.15	$\color{red}{\blacktriangle}$ +0.11	92/198
agentlessCodeCoverage	46.14	46.30	$\color{green}{\blacktriangledown}$ -0.16	45.38	$\color{red}{\blacktriangle}$ +0.76	92/197
agentlessLineCoverage	56.89	56.55	$\color{red}{\blacktriangle}$ +0.34	56.55	$\color{red}{\blacktriangle}$ +0.34	92/197

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - pass4s

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	8.97	10.54	$\color{green}{\blacktriangledown}$ -1.57	10.13	$\color{green}{\blacktriangledown}$ -1.16	92/224
agentless	11.80	9.92	$\color{red}{\blacktriangle}$ +1.88	9.92	$\color{red}{\blacktriangle}$ +1.88	91/194
agentlessCodeCoverage	17.91	17.38	$\color{red}{\blacktriangle}$ +0.53	17.73	$\color{red}{\blacktriangle}$ +0.18	82/178

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - reactive-streams-jvm

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	22.19	22.54	$\color{green}{\blacktriangledown}$ -0.35	22.54	$\color{green}{\blacktriangledown}$ -0.35	122/257
agentless	19.06	19.59	$\color{green}{\blacktriangledown}$ -0.53	19.59	$\color{green}{\blacktriangledown}$ -0.53	117/220
agentlessCodeCoverage	21.68	20.39	$\color{red}{\blacktriangle}$ +1.29	20.39	$\color{red}{\blacktriangle}$ +1.29	118/220
agentlessLineCoverage	28.04	27.52	$\color{red}{\blacktriangle}$ +0.52	27.52	$\color{red}{\blacktriangle}$ +0.52	79/154

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - heliboard

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	9.80	9.73	$\color{red}{\blacktriangle}$ +0.07	9.92	$\color{green}{\blacktriangledown}$ -0.12	89/194

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-kotlin

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	12.91	13.67	$\color{green}{\blacktriangledown}$ -0.76	13.40	$\color{green}{\blacktriangledown}$ -0.49	104/239
agentless	11.97	12.37	$\color{green}{\blacktriangledown}$ -0.40	12.37	$\color{green}{\blacktriangledown}$ -0.40	104/209
agentlessCodeCoverage	15.86	15.72	$\color{red}{\blacktriangle}$ +0.14	15.72	$\color{red}{\blacktriangle}$ +0.14	104/209
agentlessLineCoverage	17.62	17.38	$\color{red}{\blacktriangle}$ +0.24	17.73	$\color{green}{\blacktriangledown}$ -0.11	102/207

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - jolokia

Job Status: 🟢 success

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	96.16	97.04	$\color{green}{\blacktriangledown}$ -0.88	95.12	$\color{red}{\blacktriangle}$ +1.04	107/241
agentless	91.37	91.39	$\color{green}{\blacktriangledown}$ -0.02	91.39	$\color{green}{\blacktriangledown}$ -0.02	107/213
agentlessCodeCoverage	100.20	101.00	$\color{green}{\blacktriangledown}$ -0.80	99.00	$\color{red}{\blacktriangle}$ +1.20	107/213
agentlessLineCoverage	102.09	103.04	$\color{green}{\blacktriangledown}$ -0.95	101.00	$\color{red}{\blacktriangle}$ +1.09	117/229

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[datadog-prod-us1-5]

🎯 Code Coverage (details)
• Patch Coverage: 66.67%
• Overall Coverage: 59.94% (+0.02%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 144197a | Docs | Give us feedback!}

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - okhttp

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	18.95	18.82	$\color{red}{\blacktriangle}$ +0.13	19.59	$\color{green}{\blacktriangledown}$ -0.64	107/240
agentless	18.29	18.45	$\color{green}{\blacktriangledown}$ -0.16	19.20	$\color{green}{\blacktriangledown}$ -0.91	106/215
agentlessCodeCoverage	21.44	21.65	$\color{green}{\blacktriangledown}$ -0.21	22.54	$\color{green}{\blacktriangledown}$ -1.10	108/219
agentlessLineCoverage	37.40	37.91	$\color{green}{\blacktriangledown}$ -0.51	38.67	$\color{green}{\blacktriangledown}$ -1.27	105/213

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - spring_boot

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	10.00	16.36	$\color{green}{\blacktriangledown}$ -6.36	16.36	$\color{green}{\blacktriangledown}$ -6.36	104/237
agentless	10.54	9.92	$\color{red}{\blacktriangle}$ +0.62	9.92	$\color{red}{\blacktriangle}$ +0.62	104/209
agentlessCodeCoverage	14.43	13.94	$\color{red}{\blacktriangle}$ +0.49	13.67	$\color{red}{\blacktriangle}$ +0.76	104/211
agentlessLineCoverage	23.58	22.99	$\color{red}{\blacktriangle}$ +0.59	22.54	$\color{red}{\blacktriangle}$ +1.04	104/209

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[dd-octo-sts]

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.71 s	14.68 s	[-0.6%; +1.1%] (no difference)
startup:insecure-bank:tracing:Agent	13.45 s	13.56 s	[-1.5%; -0.0%] (maybe better)
startup:petclinic:appsec:Agent	17.06 s	16.93 s	[-0.0%; +1.6%] (no difference)
startup:petclinic:iast:Agent	16.27 s	17.01 s	[-8.5%; -0.2%] (maybe better)
startup:petclinic:profiling:Agent	16.70 s	16.68 s	[-1.0%; +1.2%] (no difference)
startup:petclinic:sca:Agent	17.15 s	16.89 s	[+0.5%; +2.5%] (maybe worse)
startup:petclinic:tracing:Agent	16.10 s	16.20 s	[-1.6%; +0.3%] (no difference)

Commit: 144197a2 · CI Pipeline · Benchmarking Platform UI

---

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

[cit-pr-commenter-54b7da]

CI Visibility Test Environment - sonar-java

Job Status: 🔴 failed

Scenario	This PR (%)	7d median	Δ 7d	30d median	Δ 30d	runs (7d/30d)
agent	31.18	14.51	$\color{red}{\blacktriangle}$ +16.67	14.23	$\color{red}{\blacktriangle}$ +16.95	104/237
agentless	58.75	12.37	$\color{red}{\blacktriangle}$ +46.38	12.87	$\color{red}{\blacktriangle}$ +45.88	104/211
agentlessCodeCoverage	83.78	84.36	$\color{green}{\blacktriangledown}$ -0.58	81.05	$\color{red}{\blacktriangle}$ +2.73	104/211
agentlessLineCoverage	182.05	120.92	$\color{red}{\blacktriangle}$ +61.13	123.36	$\color{red}{\blacktriangle}$ +58.69	104/212

Baseline: median of @test.tracer_overhead on main (gitlab) over the last 7/30 days, per OSS project & scenario. Δ = this PR − baseline median; red ▲ = more overhead, green ▽ = less overhead than baseline.

Tests run to verify that CI Visibility behavior has not regressed in the current PR.

[chatgpt-codex-connector]

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	✅ Completed 2026-10-19T22:13:08.551976Z	5594093	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

[datadog-prod-us1-5]

More details

Protected Spock features now supply declaration metadata during whole-spec skip aggregation, preventing backend disabling from bypassing their exclusion policy. The completed static review found no reportable defect.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 5594093}

[datadog-prod-us1-5]

Inherited Spock features ignore exclusion annotations, allowing disabling, failure suppression, and retries despite protection. The shared TestNG fixture also fails on supported older versions before its compatibility assumption can skip the scenario.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 0f5965a}

[calvinbayer]

@autotest Review

[datadog-prod-us1-5]

Inherited feature lookup can apply another feature's automation exclusions and combine parent-method line numbers with a subclass source file, causing incorrect policy decisions and source metadata.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit b5f7cc1}

[calvinbayer]

@autotest Review

[datadog-prod-us1-5]

More details

Spock feature and iteration metadata supports the declaration lookup used for exclusion checks, including inherited features. Source resolution consistently follows the method’s declaring class.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit c9978b6}

[calvinbayer]

@autotest Review

[datadog-prod-us1-5]

More details

Spock declaration metadata flows consistently through skip checks, execution policies, and reporting, including inherited features and data-driven iterations.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 144197a}

[ygree]

This promises exclusions for every data-driven iteration, but the new fixtures contain only non-parameterized features.

[calvinbayer]

Agreed: the new exclusion fixtures cover ordinary and inherited features, but not a data-driven protected feature. Iteration nodes retain the feature declaration used by skip checks, execution policies, and reporting; I verified that metadata in the Spock 2.0, 2.4, and 2.5 artifacts. I have not reproduced a behavior defect here. The missing direct regression is a coverage gap, which I am flagging for discussion rather than changing code automatically.

[ygree]

IDM approved