Skip to content

Document test automation exclusion annotations - #40827

Open
calvinbayer wants to merge 4 commits into
masterfrom
codex/test-automation-exclusions
Open

calvinbayer wants to merge 4 commits into
masterfrom
codex/test-automation-exclusions

Conversation

@calvinbayer

@calvinbayer calvinbayer commented Oct 19, 2026 •

Copy link
Copy Markdown
Contributor

What does this PR do? What is the motivation?

Documents code annotations for protecting individual tests from Datadog failure suppression and automatic retries. Adds JavaScript/TypeScript, Java, and Python usage sections with framework-specific examples and limits, and links the guide from Flaky Tests Management, Auto Test Retries, Early Flake Detection, and the guides index.

Merge readiness

  • Ready for merge

AI assistance

Codex drafted the changes and checked the examples and behavior against the SDK implementations.

For Datadog employees:

  • The branch follows the required <name>/<description> convention.

Additional notes

Confirm the corresponding SDK releases and minimum supported versions before merging this PR. Related implementation stacks: JavaScript, Java, and pytest.

Validation: no Vale alerts on new content; existing alerts are outside changed lines. YAML and Markdown parse for all five pages, all ten guide links resolve locally, JavaScript examples pass Node syntax checks, and the Python example passes AST parsing. Java examples were checked against the public APIs. Repository staged-content hooks and whitespace checks pass. A full Hugo site build was not run.

@calvinbayer
calvinbayer marked this pull request as ready for review October 19, 2026 22:10
@calvinbayer
calvinbayer requested review from a team as code owners October 19, 2026 22:10

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: WARN

The guide omits minimum Datadog SDK versions for the new annotation APIs, leaving existing users unable to determine whether the examples are supported.

Open Bits AI session

🤖 Bits Code Review · Commit ba50cb9

Comment thread hugo/content/en/tests/guides/test_automation_exclusions.md
@calvinbayer
calvinbayer requested a review from a team as a code owner October 19, 2026 22:21
@OliviaShoup OliviaShoup added the editorial review Waiting on a more in-depth review label Oct 19, 2026
@github-actions github-actions Bot added the Architecture Everything related to the Doc backend label Oct 19, 2026

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The guide explicitly marks all three annotation APIs as unreleased, avoiding unsupported compatibility claims. SDK runtime behavior and full Hugo rendering remain unverified.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit a9beae3

@OliviaShoup

Copy link
Copy Markdown
Contributor

Thanks for the the PR! Created editorial Jira ticket DOCS-16061 so someone on the docs team can take a look soon.

@calvinbayer

Copy link
Copy Markdown
Contributor Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The guide explicitly requires development SDK builds because the annotation APIs are unreleased. SDK execution behavior and full-site rendering remain independently unverified.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit f710b53

@calvinbayer

Copy link
Copy Markdown
Contributor Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The guide explicitly marks the annotation APIs as unreleased and requires compatible development builds. The completed static review found no actionable defects, though SDK execution behavior and full-site rendering remain unverified.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit 2b53be3

@calvinbayer
calvinbayer force-pushed the codex/test-automation-exclusions branch from 2b53be3 to caeb3b9 Compare October 20, 2026 18:43
@calvinbayer

Copy link
Copy Markdown
Contributor Author

@autotest Review

@datadog-datadog-prod-us1 datadog-datadog-prod-us1 Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bits Code Review: PASS

More details

The guide explicitly limits the annotation APIs to unreleased SDK development builds and documents framework-specific restrictions. SDK runtime compatibility remains unverified.

Was this helpful? React 👍 or 👎

Open Bits AI session

🤖 Bits Code Review · Commit caeb3b9

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Architecture Everything related to the Doc backend editorial review Waiting on a more in-depth review

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants