I have a question about retrieving an attribute in Grafana from Jaeger.
The java code (simplified) that produces the span is as following:
Span span = tracer.spanBuilder(“sinewave”).startSpan();
Thread.sleep((long) (20 + rnd.nextDouble()50));
sineValue = Math.sin(new Date().getTime() * Math.PI2 / 1000 / 7); // every 7 seconds
span.setAttribute(“value”, sineValue);
span.end();
I’m successful in saving the traces in Jaeger and ElasticSearch using Otel collector.
Using ElasticSearch source i had no problem in plotting the “value” field extracted from the span, where I see the sinewave.
However using the Jaeger source, the only plottable value is the duration of the segment.
The value is even there, once inspecting the Jaeger query, the information is there:
traceID:“e0159cd5ac88fc7721eaf931256168cb”
spans:Array[1]
0:Object
traceID:“e0159cd5ac88fc7721eaf931256168cb”
spanID:“7be2236a5b0b2e43”
operationName:“coswave”
references:Array[0]
startTime:1678911821187524
duration:46972
tags:Array[4]
0:Object
1:Object
2:Object
3:Object
key:“value”
type:“float64”
value:-0.03351019606799947
The value does not make it to the “Data” panel. The only fields are TraceID, Trace Name, StartTime and Duration.
Can anyone tell how can I use the Jaeger source to plot the “value” tag, and not just the duration of the span?
Thank you,
Razvan
2 Likes
Did you ever find and answer to this?
Hello Denise,
No, nobody replied. Also, I was just playing with a dashboard of an IoT device, otherwise I could have looked in the source code and add the fields myself.
Thank you,
Razvan
My work around for this, was to use the span metrics processor which I could used to get metrics info on my traces, which is what I was looking for. IE, how many times in my traces did I see and error or exception.
I’ve noticed that Metric and Spans are two different “universes” in Grafana. Maybe because the metrics and spans were unified quite recently.
Metrics seem to have a much better support.
I was monitoring a robot controls, for example Motor X has run with power Y between timestamps t1 and t2. In theory, a Span can be open, meaning the motor started at time t3 and still runs. This happens if you have a linear motor (drone, rover) and the current motor level is sufficient for the desired move, or when the joint is stuck.
Still, I don’t know if Grafana is the best tool to achieve that. Also, the underlining storage (Jaeger, Prometheus or Elastic) introduces significant delay. The ideal tool shall read the UDP broadcast of the sensors for real time data and buffer it, but rely on the storage for historical data.
Theres is more “universes”: metrics, traces (not spans), logs. These are different types of signals.
Make sure you are using right signal type for your case. IMHO trace is not right signal for your case - why you need trace, when it looks like a metric?
You also complain about delay - why you don’t use streaming?
