Hi,
I am trying to generate and embed a Tempo query in Grafana alert. Unfortunately Labels and annotations docs aren’t very helpful proiding very basic dry examples. So far I did manage to construct a basic link taking me to Tempo Explore UI in Grafana, but when I am adding more to the query then I am getting nothing.
Here is my piece of Terraform code:
resource "grafana_rule_group" "prometheus_response_time_metrics_rule_group" {
for_each = var.deployment_name
name = join(" ", ["Service endpoint response high for", each.value["deployment_name"]])
folder_uid = grafana_folder.prometheus_rule_folder.uid
interval_seconds = 300
org_id = 1
rule {
name = join(" ", ["Service response time high for:", each.value["deployment_name"]])
for = "5m"
condition = "C"
no_data_state = "OK"
exec_err_state = "Alerting"
annotations = {
__dashboardUid__ = grafana_dashboard.prometheus_otel_response_time.uid
__panelId__ = "1"
"description" = "{{ with $values }} The P75 response time is higher than 6.4 secs. for last 15 minutes, current response time is: {{ humanizeDuration $values.B.Value }} {{ end }}"
"traces" = <<EOT
${var.grafana_url}{{ tableLink "{\"expr\": \"up\", \"datasource\": \"Tempo\", \"queryType\": \"traceql\", \"query\": \"{kind=server && resource.service.name=\\\"$labels.service\\\" && name=\\\"$labels.span_name\\\" && duration>6s}\"}" }}
EOT
}
labels = {
"alert_label" = "outages"
}
...
Here is the TraceQL query I’d like to embed in the Slack alert:
{kind=server && resource.service.name="service_name" && name="span_name" && duration>6s}
Is there anywhere a more detailed documentation with various examples?