Export propagation tags in OTLP trace export by anna-git · Pull Request #12985 · DataDog/dd-trace-java

anna-git · 2026-10-21T15:57:02Z

What Does This Do

Exports the span propagation tags (_dd.p.dm, _dd.p.ksr, ...) and baggage-as-tags items as string attributes in OTLP trace export (protobuf and JSON), mirroring the Datadog mappers:

OtlpTraceProto / OtlpTraceJson MetaWriters now write Metadata.getBaggage(), which was computed but never serialized. Span tags, and the attributes the writers emit directly (resource.name, operation.name, thread.name, _sampling_priority_v1, ...), win on a key clash, so OTLP attribute keys stay unique.
Propagation tags (ROOT_PROPAGATION_TAGS, read from the local root's context) are written once per trace in the payload, on the span the collector already selects for sampling tags (first exported span of a trace in protobuf, last in JSON). This covers partial flushes whose chunk has no local root span; other spans don't carry _dd.p.*. The Agent takes _dd.p.dm from the first span in a chunk that has it, so the carrier doesn't need to be the root.
_dd.p.tid is intentionally not exported: the OTLP traceId already carries all 128 bits, and the Agent's OTLP receiver derives the upper 64 bits from it, so the tag would be redundant (Java generates 128-bit trace ids by default, so every root span would otherwise carry it). Same as the .NET change.

Motivation

With DD_TRACE_OTEL_EXPORTER=otlp, remote and adaptive sampling rules are applied (priority, _dd.rule_psr are exported), but the sampling decision maker _dd.p.dm never was. The Agent then falls back to a default decision maker, so the backend doesn't count the traffic as rule/adaptive-sampled and adaptive sampling can't converge for the service.

This complements DataDog/datadog-agent#58471, which preserves _dd.p.dm from OTLP span attributes (or t.dm from a tracestate dd= member). The .NET equivalent is DataDog/dd-trace-dotnet#9498.

Additional Notes

Validated with a local DDOT collector and remote/adaptive sampling rules: before, traces reached intake with _dd.p.dm:-4; with this change the exported trace is expected to carry the rule's decision maker (-11 remote user rule / -12 remote adaptive rule), which the Agent keeps with [trace-agent] Preserve tracer sampling decision maker in OTLP receiver datadog-agent#58471. The end-to-end run with this exact build is still to do.
Unit tests in OtlpTraceProtoTest and OtlpTraceJsonCollectorTest: propagationTagsAreWrittenOncePerTraceOnTheSpanSelectedForSamplingTags, partialFlushWithoutTheLocalRootStillCarriesPropagationTags and propagationTagsAreWrittenOnceForEachTraceInThePayload check exactly one copy of _dd.p.dm/_dd.p.ksr per trace, on the selected span, no _dd.p.tid, and baggage on every span; baggageDoesNotDuplicateDirectlyWrittenAttributes in both checks that baggage items named like directly emitted attributes don't duplicate them. :dd-trace-core:spotlessCheck couldn't run locally because the Eclipse p2 site used by the Groovy formatter was down; changed files were checked with google-java-format 1.36.1.
Follow-ups, not in this PR:
- Locally started traces have no dd= tracestate member, so the OTLP trace_state doesn't carry t.dm.

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Once approved, use merge queue to merge the PR

Jira ticket: N/A

🤖 Generated with Claude Code

The OTLP trace writers dropped the Metadata baggage map, which carries the propagation tags (_dd.p.dm, _dd.p.ksr, _dd.p.tid...) and baggage items injected as tags. Write it like the Datadog mappers do, with propagation tags on the local root span only, so the sampling decision maker reaches the Agent. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

dd-octo-sts · 2026-10-21T16:44:13Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	13.98 s	13.99 s	[-0.7%; +0.7%] (no difference)
startup:insecure-bank:tracing:Agent	12.88 s	13.00 s	[-1.6%; -0.1%] (maybe better)
startup:petclinic:appsec:Agent	17.77 s	17.53 s	[+0.3%; +2.5%] (maybe worse)
startup:petclinic:iast:Agent	17.41 s	17.61 s	[-2.0%; -0.3%] (maybe better)
startup:petclinic:profiling:Agent	17.29 s	17.28 s	[-1.1%; +1.2%] (no difference)
startup:petclinic:sca:Agent	17.70 s	17.63 s	[-0.5%; +1.3%] (no difference)
startup:petclinic:tracing:Agent	16.60 s	16.81 s	[-2.2%; -0.4%] (maybe better)

Commit: 8c0d93d8 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

The OTLP traceId already carries all 128 bits, so the tag is redundant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

A public Tags constant must be registered in the tag registry; the key is only needed by the OTLP writers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot

🟡 Changes recommended

Both encoders can emit duplicate attribute keys when baggage collides with directly emitted span fields or metadata.

1 open issue

Prevent duplicate OTLP attributes from baggage collisions · New

What changed in this PR

This PR adds propagation tags and baggage-as-tags to Java’s OTLP trace export, allowing downstream Agent processing to retain sampling decision metadata.

Changes:

Serialize baggage and propagation tags in protobuf and JSON, with collision checks against span tags.
Export propagation tags only on local roots and omit redundant _dd.p.tid.
Add tests for sampling tags, inherited baggage, and root-only propagation tags.

File	Description
dd-trace-core/src/test/java/datadog/trace/core/otlp/trace/OtlpTraceProtoTest.java	Adds protobuf export assertions and attribute parsing helpers.
dd-trace-core/src/test/java/datadog/trace/core/otlp/trace/OtlpTraceJsonCollectorTest.java	Adds JSON export assertions and span lookup helper.
dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceProto.java	Serializes baggage and root propagation tags into protobuf attributes.
dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceJson.java	Applies equivalent attribute serialization to JSON export.

_{Review effort: Balanced}

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

chatgpt-codex-connector · 2026-10-23T14:49:18Z

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review	Status	Commit	Review trigger
📝 Code Review	🔄 Running since 2026-10-23T14:49:13.697252Z	`6d518d3`	Draft marked ready

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

datadog-datadog-prod-us1

Baggage collision filtering misses direction-dependent OTLP tag renames. With baggage injection and OTel semantics enabled, both encoders can emit conflicting server.port attributes instead of preserving the span's value.

_{Open Bits AI session}

_{🤖 Bits Code Review · Commit 6d518d3}

mcculls

Added some proposed patches (drafted with codex) - once those are fixed then I'll re-review

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Copilot

🟡 Changes recommended

With baggage injection enabled and OTel semantics disabled, alias-aware lookup in both OTLP encoders incorrectly suppresses baggage whose exported key differs from the span tag’s.

1 open issue

Fix baggage suppression when aliased span tag keys differ · New

1 resolved since last review

Prevent duplicate OTLP attributes from baggage collisions

_{Review effort: Balanced}

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mcculls

Thanks for the refactoring!

mcculls · 2026-10-28T09:53:33Z

Make sure to merge in latest origin/master to pick up the OTel semantic refactoring from @zacharycmontoya (it should merge cleanly)

mcculls · 2026-10-28T10:16:48Z

Also could you coordinate with @mhlidd since this overlaps slightly with #13065, basically agree which one will get merged first as the other will need to resolve any conflicts.

…rt-propagation-tags

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

dd-octo-sts · 2026-10-28T14:21:01Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-10-28T14:21:05Z

View all feedbacks in Devflow UI.

2026-10-28 14:21:05 UTC ℹ️ Start processing command /merge

2026-10-28 14:21:10 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in master is approximately 2h (p90).

2026-10-28 15:22:07 UTC ℹ️ MergeQueue: This merge request was merged

dd-octo-sts · 2026-10-28T15:15:31Z

/merge

gh-worker-devflow-routing-ef8351 · 2026-10-28T15:15:36Z

View all feedbacks in Devflow UI.

2026-10-28 15:15:35 UTC ℹ️ Start processing command /merge

2026-10-28 15:15:38 UTC ❌ MergeQueue

PR already in the queue with status in_progress

anna-git added type: bug fix Bug fix comp: core Tracer core tag: ai generated Largely based on code generated by an AI or LLM labels Oct 21, 2026

This comment has been minimized.

Sign in to view

anna-git and others added 2 commits October 22, 2026 13:54

Don't export _dd.p.tid as an OTLP attribute

900a307

The OTLP traceId already carries all 128 bits, so the tag is redundant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Keep the _dd.p.tid key local to the OTLP writers

6dce281

A public Tags constant must be registered in the tag registry; the key is only needed by the OTLP writers. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anna-git requested a balanced review from Copilot October 23, 2026 12:21

Copilot started reviewing on behalf of anna-git October 23, 2026 12:22

View session

Copilot AI reviewed Oct 23, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceProto.java Resolved

anna-git and others added 2 commits October 23, 2026 15:53

Don't let baggage duplicate directly emitted OTLP attributes

a0096d2

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Share the OTLP span attribute keys between the trace writers

6d518d3

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anna-git marked this pull request as ready for review October 23, 2026 14:49

anna-git requested a review from a team as a code owner October 23, 2026 14:49

anna-git requested review from khanayan123 and removed request for a team October 23, 2026 14:49

anna-git requested a review from mcculls October 23, 2026 14:57

datadog-datadog-prod-us1 Bot reviewed Oct 23, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceProto.java Resolved

mcculls reviewed Oct 23, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceJson.java Resolved

mcculls reviewed Oct 23, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceProto.java Resolved

mcculls reviewed Oct 23, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceJson.java Resolved

mcculls reviewed Oct 23, 2026

View reviewed changes

anna-git and others added 2 commits October 26, 2026 15:00

Export propagation tags on partial flushes without the local root

ee1fa5b

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

Don't let baggage duplicate direction-dependent OTLP attribute names

6dfd8ee

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anna-git requested a balanced review from Copilot October 27, 2026 13:14

Copilot started reviewing on behalf of anna-git October 27, 2026 13:18

View session

Copilot AI reviewed Oct 27, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceProto.java Resolved

Skip baggage only when a span tag is exported under the same OTLP key

7212905

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anna-git requested a review from mcculls October 27, 2026 15:12

Keep the alias TableTest rows on one line for spotless

359e585

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

mcculls reviewed Oct 28, 2026

View reviewed changes

Comment thread dd-trace-core/src/main/java/datadog/trace/core/otlp/trace/OtlpTraceAttributes.java Resolved

mcculls approved these changes Oct 28, 2026

View reviewed changes

mcculls requested a review from dougqh October 28, 2026 09:54

mcculls mentioned this pull request Oct 28, 2026

Write global tags once on the OTLP trace resource #13065

Merged

anna-git and others added 2 commits October 28, 2026 13:34

Merge remote-tracking branch 'origin/master' into anna.yafi/otlp-expo…

0ff0c9e

…rt-propagation-tags

Rename OtlpTraceAttributes to OtlpSpanAttributes

8c0d93d

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

anna-git added this pull request to the merge queue Oct 28, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 28, 2026

anna-git added this pull request to the merge queue Oct 28, 2026

github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Oct 28, 2026

gh-worker-dd-mergequeue-cf854d Bot merged commit ab5833d into master Oct 28, 2026
623 checks passed

gh-worker-dd-mergequeue-cf854d Bot deleted the anna.yafi/otlp-export-propagation-tags branch October 28, 2026 15:22

github-actions Bot added this to the 1.69.0 milestone Oct 28, 2026

mhlidd mentioned this pull request Oct 28, 2026

Keep traces with different global tags in one OTLP payload rather than eagerly flushing #13112

Merged

Conversation

anna-git commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Oct 21, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

chatgpt-codex-connector Bot commented Oct 23, 2026

Codex Review Summary

Uh oh!

datadog-datadog-prod-us1 Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mcculls left a comment

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

🟡 Changes recommended

Uh oh!

Uh oh!

Uh oh!

mcculls left a comment

Choose a reason for hiding this comment

Uh oh!

mcculls commented Oct 28, 2026

Uh oh!

mcculls commented Oct 28, 2026

Uh oh!

dd-octo-sts Bot commented Oct 28, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

dd-octo-sts Bot commented Oct 28, 2026

Uh oh!

gh-worker-devflow-routing-ef8351 Bot commented Oct 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

anna-git commented Oct 21, 2026 •

edited

Loading

dd-octo-sts Bot commented Oct 21, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Oct 28, 2026 •

edited

Loading

gh-worker-devflow-routing-ef8351 Bot commented Oct 28, 2026 •

edited

Loading