How to configure Service Graphs with Grafana Tempo without having a metrics backend?

Hi team,

I wanted to know if it is possible to get service graphs in Grafana with Tempo as data source ? I checked this document and it mentions the use of prometheus, but is it possible to configure service graphs without having Prometheus configured ?

Thank you for the help !

At the moment you do have to have a backend that supports both Prometheus remote write and PromQL.

Remote Write:

PromQL:

We do intend to eventually support metrics from traces directly in Tempo, but that work is likely a ways off. Perhaps service graphs could be queried directly from Tempo at that time.

Hi,

Is the other way around possible? eg. we have only ts metrics with necessary labels (
service_graph_request_total{server=a,…}) on Prometheus and Prometheus as data source on node graph panel.

:slight_smile: Cheers,
:Devrim

Hi @devrimdemiroz, could you expand on what you want to achieve, and what your set-up is?

The short answer is that the metrics-generator needs to remote write its metrics to a Prometheus-compatible backend.

Is it possible to get service graphs in Grafana with prometheus datasource (without tempo dependency)?
Assume that, we put metrics in prometheus with all necessary labels already like , server, client, connection_type.

Service graphs are a visualisation of the Tempo datasource only. But you can configure any datasource without the source actually working, including Tempo. Then, you just need to add the configuration needed for the service graphs.

Thank you! I imagined a usage as NodeGraph add support for SQLite · Discussion #48629 · grafana/grafana · GitHub a nodegraph with two promql queries as otelcol-servicegraphprocessor might now be possible…

1 Like

Now possible :innocent: promql’s do work at up to a certain level od visuals after 9.1. However a fake Tempo resource does it much better. Here is how.

Nice write up! The initial idea was for Tempo and OTel metrics to always be consistent so that what you’re doing would be way easier. As you’ve discovered things have drifted over time :).

We are currently working to correct this, but in the meantime you’ve put together an awesome solution.

1 Like

@joeelliott , as you said so, I just put together to get it visible. The real value is yours. Btw. just improvising, wouldn’t it be nice to have a topology processor? :innocent:

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.