Skip to content

Add agentless and standalone OpenFeature smoke tests - #12846

Draft
PerfectSlayer wants to merge 2 commits into
bbujon/ffe-parityfrom
bbujon/ffe-extras
Draft

PerfectSlayer wants to merge 2 commits into
bbujon/ffe-parityfrom
bbujon/ffe-extras

Conversation

@PerfectSlayer

@PerfectSlayer PerfectSlayer commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

What Does This Do

Extracts the shared OpenFeature smoke test scenarios into AbstractOpenFeatureSmokeTest and runs them in three setups: SDK only (standalone), SDK + agent in agentless mode, and SDK + agent with Remote Configuration. Updates ARCHITECTURE.md, and adds the dd-openfeature SDK README: setup, configuration, features by setup, migration from earlier versions and design notes for contributors.

Motivation

Covers each supported setup of the standalone SDK (APMLP-1512).

Stack

The full change is in #12709. It was split into this stack to ease the review.

  1. Resolve Feature Flags enablement from instrumenter config #12845 (merged)
  2. Add dd-openfeature SDK flag configuration and evaluation core #12840 (merged)
  3. Add dd-openfeature SDK settings and configuration delivery #12841 (merged)
  4. Add dd-openfeature SDK exposure and flag evaluation pipelines #12842 (merged)
  5. Add dd-openfeature SDK OpenFeature provider #12843 (merged)
  6. Port master Feature Flags changes to the dd-openfeature SDK #12957 (merged)
  7. Add the agent bridge for the dd-openfeature SDK #12975
  8. Connect the dd-openfeature SDK to the agent #12844
  9. Keep released dd-openfeature SDKs working with the agent #12976
  10. Resolve the Feature Flags settings like the SDK #12977
  11. Restore the agent Feature Flags behaviors through the bridge #12978
  12. Add agentless and standalone OpenFeature smoke tests #12846 👈 this PR

Jira ticket: APMLP-1512

🤖 Generated with Claude Code

@datadog-official

datadog-official Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 59.34% (+0.01%)

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1aee4c8 | Docs | Give us feedback!

@dd-octo-sts

dd-octo-sts Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite Status
Startup 🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results
Scenario Candidate master Δ (95% CI of mean)
startup:insecure-bank:iast:Agent 14.07 s 14.01 s [-0.4%; +1.3%] (no difference)
startup:insecure-bank:tracing:Agent 12.93 s 12.94 s [-0.8%; +0.7%] (no difference)
startup:petclinic:appsec:Agent 17.21 s 16.56 s [-0.4%; +8.3%] (no difference)
startup:petclinic:iast:Agent 17.07 s 17.11 s [-0.9%; +0.5%] (no difference)
startup:petclinic:profiling:Agent 16.87 s 16.83 s [-0.7%; +1.0%] (no difference)
startup:petclinic:sca:Agent 17.10 s 17.00 s [-0.6%; +1.7%] (no difference)
startup:petclinic:tracing:Agent 15.79 s 16.12 s [-6.3%; +2.2%] (no difference)

Commit: 1aee4c88 · CI Pipeline · Benchmarking Platform UI


Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

@gh-worker-dd-mergequeue-cf854d
gh-worker-dd-mergequeue-cf854d Bot force-pushed the bbujon/ffe-extras branch 2 times, most recently from 3c9ff95 to 272d58c Compare October 16, 2026 18:02
@PerfectSlayer
PerfectSlayer removed this pull request from stack #12847 October 20, 2026 11:30
@PerfectSlayer
PerfectSlayer added this pull request to stack #12958 October 20, 2026 11:31
@PerfectSlayer
PerfectSlayer force-pushed the bbujon/ffe-extras branch 2 times, most recently from dc45c3e to 0aef5b5 Compare October 20, 2026 12:40
@sarahchen6
sarahchen6 force-pushed the bbujon/ffe-extras branch 3 times, most recently from 442e327 to 17833c4 Compare October 21, 2026 02:16
@PerfectSlayer
PerfectSlayer force-pushed the bbujon/ffe-extras branch 2 times, most recently from 5f96f72 to 716f7a1 Compare October 21, 2026 09:13
PerfectSlayer and others added 2 commits October 22, 2026 13:38
Extract the shared OpenFeature smoke test scenarios and run them against
the SDK alone, the SDK with the agent in agentless mode, and the SDK
with the agent through Remote Configuration. Document the new SDK
layout in ARCHITECTURE.md.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Describe the SDK setup, its configuration, the features by setup, the
migration from earlier versions and the design notes for contributors.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

comp: openfeature OpenFeature tag: ai generated Largely based on code generated by an AI or LLM tag: no release notes Changes to exclude from release notes tag: override groovy enforcement Override the "Enforce Groovy Migration" check type: feature Enhancements and improvements

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant