A more elegant way to propagate tracing context
In previous posts,[1] I mentioned that in Rust, tracing is often used alongside opentelemetry to build a local-cluster model. In this setup, tracing is responsible for generating and structuring local spans, while opentelemetry takes care of propagating context across service boundaries.
However, I’ve come to realize that this explanation alone doesn’t quite capture what a real-world implementation of tracing injection looks like.
In this post, I’ll walk you through a more elegant approach to integrating tracing
with opentelemetry
. This method goes beyond simply extracting metadata—by leveraging opentelemetry::propagation
, we can achieve a much cleaner and more robust solution.
The problem
In complex applications, remote calls are common. In a microservices architecture, services communicate with each other through RPC. In message queue models, we often need to propagate context between the publish and consume stages. In these scenarios, it's crucial to pass the tracing context across process boundaries.
In the previous post, I used manual metadata extraction to propagate tracing context. This approach is verbose and error-prone. For example:
req.metadata_mut().insert(
RPC_TRACE_ID,
span.context() // get the opentelemetry context
.span() // get the opentelemetry span
.span_context() // get the opentelemetry span context, including informations that we need to transmit
.trace_id()
.to_string()
.parse()
.unwrap(),
);
This method makes us the metadata extractor, manually accessing and parsing data from the span context. Instead of doing this, the opentelemetry::propagation
module offers a more elegant solution using the Injector
and Extractor
traits. Injector
is used to insert context into a data structure, and Extractor
retrieves it. With these traits, context propagation becomes cleaner and more maintainable.
The solution
To use opentelemetry::propagation
, we need to implement the Injector
and Extractor
traits for our metadata type.
Implementation based on orphan rule
According to the orphan rule, sometimes we need to create a new type. Currently I'm working on a tracing middleware for volo-grpc
and a message wrapper for broccoli-queue
, and that's what problem I encountered.
#[derive(Debug, Clone, PartialEq, serde::Serialize, serde::Deserialize)]
pub struct MessageWithMetadata<T> {
metadata: HashMap<String, String>,
payload: T,
}
impl<T> MessageWithMetadata<T> {
pub fn new(payload: T) -> Self {
Self {
metadata: HashMap::new(),
payload,
}
}
}
impl<T> opentelemetry::propagation::Injector for MessageWithMetadata<T> {
fn set(
&mut self,
key: &str,
value: String,
) {
self.metadata.insert(key.to_string(), value);
}
}
impl<T> opentelemetry::propagation::Extractor for MessageWithMetadata<T> {
fn get(
&self,
key: &str,
) -> Option<&str> {
self.metadata.get(key).map(|s| s.as_str())
}
fn keys(&self) -> Vec<&str> {
self.metadata.keys().map(|s| s.as_str()).collect()
}
}
It’s important to note that I implemented the metadata type myself because broccoli-queue
doesn’t support complete metadata transmission out of the box. As a result, this wrapper isn’t just for enabling the propagator—it also fulfills the need to carry metadata alongside the message.
Use the Injector
and Extractor
With the Injector
and Extractor
traits implemented, we can now propagate context in a structured and type-safe way.
The TextMapPropagator
trait provides the core methods inject
and extract
for context propagation. However, you might notice that using these methods directly often results in no context being injected or extracted. This is because they rely on opentelemetry
’s internal context, which is separate from the context managed by tracing
.
Since tracing
stores context in tracing::Dispatch
, we must bridge this gap explicitly. To propagate context properly, we need to extract an opentelemetry::span::Span
from the current tracing::Span
using the tracing_opentelemetry::OpenTelemetrySpanExt
trait. We then pass the resulting opentelemetry::Context
to the appropriate propagation methods—inject_context
for injection and extract_with_context
for extraction.
The updated code looks like this:
// Inject context into the outgoing message
let cx = tracing::Span::current().context();
let mut message_wrap = MessageWithMetadata::new(payload);
opentelemetry::global::get_text_map_propagator(|propagator| {
propagator.inject_context(&cx, &mut message_wrap);
});
// Extract context from the incoming message
let mut cx = tracing::Span::current().context();
opentelemetry::global::get_text_map_propagator(|propagator| {
cx = propagator.extract(&message.payload);
});
tracing::span::Span::current().set_parent(cx);
Conclusion
In this post, I showed you how to connect tracing
with opentelemetry
in a more elegant way using the opentelemetry::propagation
module. By implementing the Injector
and Extractor
traits for our metadata type, we can easily propagate context across process boundaries without manually extracting metadata.
Comparing to the previous method, this approach is cleaner and more maintainable. RemoteSpanContext
have many fields, and we don't need to access them manually anymore. We can just use the opentelemetry::propagation
module to do that. This is a great improvement for observability in Rust applications.