dd-trace-rs

This library powers Distributed Tracing, metrics and logging. It provides OpenTelemetry API and SDK compatibility with Datadog-specific features and optimizations.

Usage

The datadog-opentelemetry crate provides an easy to use override for the rust opentelemetry-sdk.

Installation

Add to you Cargo.toml

datadog-opentelemetry = { version = "0.5.2" }

Creating traces, metrics and logs

Tracing

To trace functions, you can either use the opentelemetry crate's API or the tracing crate API with the tracing-opentelemetry bridge.

Metrics

To collect metrics, use the opentelemetry crate's Metrics API. For more details, see the Datadog OpenTelemetry Rust documentation.

Logging

• Enable with the logs feature of this crate

To collect logs, you can use the log crate with the opentelemetry_appender_log. For more details, see the Datadog OpenTelemetry Rust documentation.

Library initialization

The following examples will read datadog and opentelemetry configuration from environment variables and other available sources, initialize and set up the global providers for tracing, logging or metrics

Tracing API

Requires

• tracing-subscriber
• tracing-opentelemetry
• tracing

use opentelemetry::trace::TracerProvider;
use std::time::Duration;
use tracing_subscriber::{layer::SubscriberExt, util::SubscriberInitExt};

// This picks up env var configuration and other datadog configuration sources
let tracer_provider = datadog_opentelemetry::tracing().init();

tracing_subscriber::registry()
    .with(
        tracing_opentelemetry::layer()
            .with_tracer(tracer_provider.tracer("my_application_name")),
    )
    .init();

tracer_provider
    .shutdown_with_timeout(Duration::from_secs(1))
    .expect("tracer shutdown error");

Opentelemetry trace API

Requires

• opentelemetry with the trace feature enabled

use std::time::Duration;

// This picks up env var configuration and other datadog configuration sources
let tracer_provider = datadog_opentelemetry::tracing().init();

// Your code
// Now use standard OpenTelemetry APIs
use opentelemetry::global;
use opentelemetry::trace::Tracer;

let tracer = global::tracer("my-service");
let span = tracer.start("my-operation");
// ... do work ...

// Shutdown the tracer to flush the remaining data
tracer_provider
    .shutdown_with_timeout(Duration::from_secs(1))
    .expect("tracer shutdown error");

Opentelemetry metrics API

Requires

• the metrics feature of this crate to be enabled
• opentelemetry with the metrics feature enabled
• tokio

The metrics provider MUST be initialized within a tokio context

// Initialize metrics with default configuration
let meter_provider = datadog_opentelemetry::metrics().init();

// Use standard OpenTelemetry Metrics APIs
use opentelemetry::global;
use opentelemetry::metrics::Counter;
use opentelemetry::KeyValue;

let meter = global::meter("my-service");
let counter: Counter<u64> = meter.u64_counter("requests").build();
counter.add(1, &[KeyValue::new("method", "GET")]);

// Shutdown to flush remaining metrics
meter_provider.shutdown().unwrap();

Log API

Requires

• the logs feature of this crate to be enabled
• log
• opentelemetry-appender-log
• tokio

The logger provider MUST be initialized within a tokio context

// Initialize logs with default configuration
let logger_provider = datadog_opentelemetry::logs().init();

let otel_log_appender = opentelemetry_appender_log::OpenTelemetryLogBridge::new(&logger_provider);
log::set_boxed_logger(Box::new(otel_log_appender)).unwrap();

// Before ending the program shutdown to flush remaining logs to the collector
logger_provider.shutdown();

For more details, see the Datadog OpenTelemetry Rust documentation.

Thread context sharing

This tracer supports sharing the active context with an external (out-of-process) reader, typically the Full Host Profiler, correlating CPU profiles with spans. The implementation follows the OTel specification, making the context visible to other compatible eBPF profilers or any OTel-compliant external reader.

The thread-level context sharing is Linux-only, as per the current standard.

To enable thread-level context sharing on Linux:

1. Enable the otel-thread-ctx feature on your datadog-opentelemetry dependency.

2. Depending on the type of your crate:

   • If you're building an executable, add datadog-opentelemetry to your [build-dependencies] as well (do not enable the otel-thread-ctx feature for the build dependency):

     [build-dependencies]
     datadog-opentelemetry = { version = "0.5.2" }

     Finally, add the following line to your build.rs main function (create one if needed):

     datadog_opentelemetry::build::emit_otel_thread_ctx_build_instructions();

   • If you're building a Rust library ((r)lib), you'll have to ensure the instructions above are applied to every downstream executable that uses your library.

   • If you're building a shared dynamic library ((c)dylib), please note that while the following instructions should work in most environments, building a dynamic library with thread context support is currently experimental at best. In particular, a crate with both a binary target and a (c)dylib target won't compile with the instructions below.

     Add datadog-opentelemetry to your [build-dependencies] as well (do not enable the otel-thread-ctx feature for the build dependency):

     [build-dependencies]
     datadog-opentelemetry = { version = "0.5.2" }

     Finally, add the following line to your build.rs main function (create one if needed):

     datadog_opentelemetry::build::emit_otel_thread_ctx_dylib_build_instructions();

We are working on lifting the need for a custom build script in the future.

Configuration

Configuration can be passed either:

• Programmatically

use datadog_opentelemetry::configuration::Config;
let config = Config::builder()
    .set_service("my_service".to_string())
    .set_env("prod".to_string())
    .build();

let tracer_provider = datadog_opentelemetry::tracing()
    .with_config(config.clone())
    // this also accepts options for the Opentelemetry SDK builder
    .with_max_attributes_per_span(64)
    .init();

let metrics_provider = datadog_opentelemetry::metrics()
    .with_config(config.clone())
    .init();

let logging_provider = datadog_opentelemetry::logs()
    .with_config(config.clone())
    .init();

For advanced usage and configuration information, check out [DatadogTracingBuilder], [configuration::ConfigBuilder] and the library documentation.

• Through env variables

DD_SERVICE=my_service DD_ENV=prod cargo run

Configuration of the OpenTelemetry SDK TracerProviderBuilder can be done when initializing the library

#[derive(Debug)]
struct MySpanProcessor;
impl opentelemetry_sdk::trace::SpanProcessor for MySpanProcessor { ... }

// Custom otel tracer sdk options
datadog_opentelemetry::tracing()
    .with_max_attributes_per_span(64)
    // Custom span processor
    .with_span_processor(MySpanProcessor)
    .init();

Tracer diagnostics

The tracer reports on itself - configuration problems, transport failures, dropped spans. Where those diagnostics go depends on the application: if it has installed a tracing subscriber, they are emitted to it under the target datadog_opentelemetry; if it has not, they are printed to stdout and stderr, so a bare binary still sees them.

With a subscriber installed, two independent settings apply, and a diagnostic has to pass both:

Setting	Decides	Default
DD_LOG_LEVEL, or ConfigBuilder::set_log_level_filter	how verbose the tracer is: which diagnostics it produces at all	ERROR
the subscriber's own filter, typically RUST_LOG	which of those it keeps, and where they go	ERROR, for EnvFilter with RUST_LOG unset

Neither overrides the other: the more restrictive of the two wins, so raising one alone leaves the other in force. Both have to be raised to see anything below ERROR.

# Errors only: the tracer produces nothing below ERROR for the filter to admit.
RUST_LOG=datadog_opentelemetry=debug cargo run

# Errors only: the tracer produces DEBUG diagnostics, and the subscriber discards them.
DD_LOG_LEVEL=debug RUST_LOG=error cargo run

# DEBUG diagnostics reach the subscriber, which decides where they end up.
DD_LOG_LEVEL=debug RUST_LOG=datadog_opentelemetry=debug cargo run

Support

• MSRV: 1.87
• opentelemetry version: 0.33
• tracing-opentelemetry version: 0.34
• opentelemetry-appender-log version 0.33
• log version 0.4

Features

• metrics enabled the metrics provider
• metrics-grpc enabled the metrics provider, with GRPC OTLP export
• metrics-http enabled the metrics provider, with HTTP OTLP export
• logs enabled the log provider
• logs-grpc enabled the log provider, with GRPC OTLP export
• logs-http enabled the log provider, with HTTP OTLP export
• log-compat routes the tracer's internal diagnostics through the log facade when no tracing subscriber is available
• otel-thread-ctx enables sharing contexts with external readers (typically an eBPF profiler). Executables additionally require a build.rs calling the build helper: see the dedicated section above.