How to Format Alert Message Array

Grafana Alerting
1

Hello guys,

I’m trying to format an alert message in “Summary and Annotations” on Grafana with no success.

I’m getting the Value below in the message:

Value: [ var='B0' metric='value' labels={host.name=ipA, metric.application=appA} value=0.8333333333333334 ], [ var='B1' metric='value' labels={host.name=ipB, metric.application=appB} value=0.81 ]

I would like to format the message like:

The disks are almost full for the applications below:
appA - ipA - 83%
appB - ipB - 81%

I have tried so many ways to get the values like:

  • {{ humanizePercentage $labels.value }}
  • {{ $values.host.name }}
  • {{ $value.host.name }}
  • {{ $value.B0.labels.host.name }}
  • etc.

Someone knows how to format the message in this case?

Regards

2

Welcome @everaldorodrigo

Can you try this?

The disks are almost full for the applications below:
{{ $values.A.Labels.metric.application }} - {{ $values.A.Labels.host.name }} - {{  printf "%.0f"  $values.A.Value }}%
{{ $values.B.Labels.metric.application }} - {{ $values.B.Labels.host.name }} - {{  printf "%.0f"  $values.B.Value }}%

image
image1019×330 39 KB

1 Like
3

Thank you so much @grant2.

I added it to the Description and I’m getting the message below:

- description = The disks are almost full for the applications below:
<no value> - <no value> - %!f(<nil>)%
<no value> - <no value> - %!f(<nil>)%

I also tried looping the values and didn’t work. Like:

{{ range .Value}}<strong>{{ .Name }}</strong>: {{ .Value }} {{ end }}

{{ range .Value}}<strong>{{ $labels.host.name }}</strong>: {{ .Value }} {{ end }}

{{ range .Labels}}<strong>{{ .Name }}</strong>: {{ .Value }} {{ end }}

A loop was going to work better in my case because the number of values varies. I was trying to get a specific item first because it will be helpful in other situations. However, I couldn’t get the values in both cases.

I appreciate your help!

1 Like
4

Can you paste the text you receive when you click Preview alerts?

image
image984×325 18.2 KB

5

I’m receiving the text below

[ var='B0' metric='value' labels={host.name=ipA, metric.application=appA} value=0.8333333333333334 ], [ var='B1' metric='value' labels={host.name=ipB, metric.application=appB} value=0.8333333333333334 ], [ var='B2' metric='value' labels={host.name=ipC, metric.application=appC} value=0.8333333333333334 ]
1 Like
6

I think the problem may be due the fact that you have a dot ( . ) in the label names. Can you try the same thing with but a label like “hostname” (instead of “host.name”)?

7

Thank you, however I don’t have labels with no dot (.). And it’s going to be hard to change the labels name or add other labels :frowning:

8

I guess we will never know.

Unless something changes…

9

Hi @everaldorodrigo
Can you try
{{ $values.B0.Labels.metric.application }} and {{ $values.B0.Value }}

and
{{ $values.B1.Labels.metric.application }} and {{ $values.B1.Value }}

10

Hi! If a label has a dot in it use the index function from Go templates:

{{ index $values.B0.Labels "metric.application" }}
11

Hello
New to the new alerting system - we have tons of alerts and i really dislike the new output
How can we make it print only the relevant alerting label?

Is there a way to control how the alerts are being migrated to the unified alerting system? (we have hundreds of legacy alerts on grafana 8)

12

Hi! What do you mean by relevant?

13

Hi,

For example:

Values from alert:
[ var=‘B0’ metric=‘{app=“formulario-24-ms”}’ labels={app=formulario-24-ms} value=1 ]

Message with go template (in spanish):
Logs con presencia de ERROR logging level, revise por posibles errores. Detalle:

{{ with $values }}
{{ range $k, $v := . }}
App: {{$v.Labels.app}}
Errores por minuto: {{ $v }}
{{ end }}
{{ end }}

Warning: labels with dot give a lot of problems.

Suerte!

14

I was using this templating with GO range but since latest version, new alerting format has 2 expressions: B is a reduce (get single value from query) and C is a threshold (evaluates B).

This format is mandatory for managing silence correctly but as a result, the alert template issues value for both queries for each alerts which is not convenient.

However, using variables in the format {{ $labels.instance }} became possible again (because labels are the same between the 2 expressions.
To get value from var='B', simply use this format {{ $values.B }}.

image
image803×197 11.5 KB

15

This is much useful

16
  1. Configure an Alert Notification Channel: Set up a notification channel in Grafana that sends alerts to a webhook or external service. Grafana supports various notification channels, and you can choose one that fits your needs.
  2. Create a Script or Service: Develop a script or a small service that receives the alert data, performs the necessary transformations, and sends the formatted message to the desired destination. This script could be written in a language of your choice (e.g., Python, JavaScript).
  3. Update Grafana Notification Channel: Update the Grafana notification channel settings to use the webhook or external service created in step 2.
  4. Format the Message: In your script or service, parse the incoming data, extract the relevant information, and format the message according to your requirements.

Without knowing the specific tools and services you have in your environment, it’s challenging to provide a detailed solution. However, this approach should give you the flexibility to format alert messages as needed.