I wanted to know if it is possible to get service graphs in Grafana with Tempo as data source ? I checked this document and it mentions the use of prometheus, but is it possible to configure service graphs without having Prometheus configured ?
At the moment you do have to have a backend that supports both Prometheus remote write and PromQL.
Remote Write:
PromQL:
We do intend to eventually support metrics from traces directly in Tempo, but that work is likely a ways off. Perhaps service graphs could be queried directly from Tempo at that time.
Is the other way around possible? eg. we have only ts metrics with necessary labels (
service_graph_request_total{server=a,…}) on Prometheus and Prometheus as data source on node graph panel.
Is it possible to get service graphs in Grafana with prometheus datasource (without tempo dependency)?
Assume that, we put metrics in prometheus with all necessary labels already like , server, client, connection_type.
Service graphs are a visualisation of the Tempo datasource only. But you can configure any datasource without the source actually working, including Tempo. Then, you just need to add the configuration needed for the service graphs.
Nice write up! The initial idea was for Tempo and OTel metrics to always be consistent so that what you’re doing would be way easier. As you’ve discovered things have drifted over time :).
We are currently working to correct this, but in the meantime you’ve put together an awesome solution.
@joeelliott , as you said so, I just put together to get it visible. The real value is yours. Btw. just improvising, wouldn’t it be nice to have a topology processor?