Grafana 11.3: how to make more labels available for alert notification?

Grafana Alerting
1

In alert rule query (Lucene) i have keys “labels.plan_name”, “labels.sql_cmd” and “labels.req_duration”.

I would like to utilize all 3 keys when sending customized alert notifications to MS Teams channel.

However, when i debug the available labels in “index .Labels”, only “labels.plan_name” is found.

My understanding is that only “labels.plan_name” is available because thats the only label out of my mentioned 3 which is shown in Grafana Alert rules’ sections B (Reduce) and C (Threshold).

Is there a way to make “labels.sql_cmd” and “labels.req_duration” visible in notification template?

2

You are correct.

To be included in Alert.Labels, labels.sql_cmd and labels.req_duration should be part of the alert rule final output, the threshold. These are available for use in notification templates.

Alternatively, you can read query labels in alert rule templates via $labels.

Consider creating a custom annotation (or custom label):

  • my_sql_cmd → {{ $labels.sql_cmd }}

Then, access the custom annotation in the notification template as usual:

{{ range .Alerts }}
  {{ .Annotations.my_sql_cmd }}
{{end}}

Hope it helps!

3

Can you share a screenshot of the query preview? Expressions do not add\drop the labels returned by the data source query. Even math expression when it “unions” two queries picks superset of labels of a dimension.

You need to make sure that the data source query returns labels that you want to include.