@jlu1 the dot is needed to provide the context.
Are you getting any notifications to slack, even a blank one?
Check out my post at the top of this thread, if you see your label shown in the alert preview output, then you can use it in the alert by following the steps there.
Yes, I receive a blank one if I use the dot.
But I couldn’t find a way of retrieving any data.
Hello,
This explanation was very helpful, thanks @melori.arellano !
I would also want to ask is there a relation between - the threshold which we set inside alert rules and the panels inside a dashboard. I am asking because I would like to display the same threshold on both graphs.
Thanks for the guide @melori.arellano, it has got me most the way but now stuck again. How are you handling long value strings or conversions to % when using these templates?
I have an AlertValue that works, but gives me a very long number value and I was trying to trim it down using one of the humanize template functions, but whenever I put the function in the AlertValue it breaks the alert.
{{ with $values }}
{{ range $k, $v := . }}
<b>Deployment:</b> {{$v.Labels.deployment}}
<b>Success Rate (%):</b> {{ $v | humanizePercentage }}
{{ end }}
{{ end }}
I have also tried it as:
{{ humanizePercentage $v }}
Also seems like the latest Grafana release has removed/broken HTML support in the annotations.
@jlu1 The problem could live in two places:
- The alert is not generating any values to add into your message template
Check the output of the alert by expanding the alert information on the alerting page, it should show the values that have been captured from the alert. Notice that it shows the annotation configuration and then when you expand the nested alert you’ll see the created labels and the expected value for the annotation:
- The message template configuration isn’t capturing the right values from the annotation in your alert. Then you’ll want to try testing the message template itself to make sure that it is properly passed to slack.
Hi @melori.arellano, you are right about the first case - I’m not receiving any data in my “check” Annotation.
Could you please help me on how to extract the “hostname.keyword” and “remoteEmail.keyword” from the Value field? Is this even possible? What would be the way of extracting those values in the case I have an array of these set of values as an output? For instance:
[ metric='value' labels={hostname.keyword=a, remoteEmail.keyword=aaa@aaa.com} value=9999 ], [ metric='value' labels={hostname.keyword=b, remoteEmail.keyword=bbb@bbb.com} value=1111 ]
Thank you very much for for help.
I tested this out on my instance using the sample data in our docs Examples of template functions
humanizePercentage
TemplateString { humanizePercentage $value }. ← this didn’t work, because in the alert template, the value was in $values.B.Value
From my alert preview, I can see that the value is in $values.B.Value
This worked for me:
{{$values.B.Value | humanizePercentage }}
1 Like
@dselena this is a good question. With the new unified alerting in Grafana 8+ alerts are decoupled from dashboards so the thresholds don’t transfer over automatically.
1 Like
How does it work with multipe values returned? I have multipe values returned which is why your initial template was so helpful. So of the queries return dozens of results e.g.
[ var=‘B0’ metric=‘server1:9198 / learnapplogs’ labels={instance=server1:9198, job=logstash_exporter, pipeline=logs} value=0.2932612419621986 ],
[ var=‘B1’ metric=‘server2:9198 / learnapplogs’ labels={instance=server2:9198, job=logstash_exporter, pipeline=logs} value=0.2856948269581056 ]
Ideally this would somehow work:
{{ with $values.B.Value | humanize }}
{{ range $k, $v := . }}
Instance: {{$v.Labels.instance}}
Pipeline: {{$v.Labels.pipeline}}
Message Count: {{ $v }}
{{ end }}
{{ end }}
1 Like
Dear @melori.arellano,
Thank you for your response! For manual set, you mean to create alert and with the same configuration for visual threshold on the panel or there is another way to connect both thresholds manually?
Thank you in advance!
So my alert Preview looks something like the following:
[ var=‘B0’ metric=‘17145’ labels={} value=4 ], [ var=‘B1’ metric=‘10040’ labels={} value=4 ], [ var=‘B2’ metric=‘Machine2’ labels={} value=4 ],
And I want to access the metric variable in the dollar sings area. How would that be possible?
{{ with $values }}
{{ range $k, $v := . }}
$$$$$$$
Alerting value: {{ $v }}
{{ end }}
{{ end }}
2 Likes
Hello @melori.arellano
Using custom labels break the HTML formatting for the email notifications in Grafana alerting. This is apparently a known issue discussed here.
Would you by any chance have any suggestions as to a potential workaround, other than what is discussed in the link above (editing the HTML template on Grafana server).
The problem we’re having is that we have data we want to suppress in the email message, such as:
Value: [ var='B' labels={instance=<XXXX> } value=53.46666666679084 ], [ var='C' labels={instance=<XXXX>} value=1 ]
Thanks!
Hi @melori.arellano is there any more help you might be able to offer here please? I’m now completely stuck on it and it feels so close to working.
1 Like
I also have the same requirement. Tried all the possible combination but cant fetch the metric value
1 Like
Hi @melori.arellano ! Is this possible for email alerts yet or is it still an issue?
If it is still a issue, is it not possible to see the values in the alert rules UI as well or just in the email body?
1 Like
I found it! I was looking through tests and notice they use .Value
{{ with $values }}
{{ range $k, $v := . }}
Location:
{{ $v.Labels.location }}
Alerting value:
{{ humanizePercentage .Value}}
{{ end }}
{{ end }}
This worked for me, I got:
2 Likes
@milinpatel13298 this thread is specific to slack, but message templates are supported for the email contact point as well.
Yes, you’d just want to create the alert using the same threshold that you provided in the dashboard.
Hi @melori.arellano! Thank you for your reply.
Would you be able to help me get started with that please?
Here’s what I have tried so far, none of these seem to work.
{{ $values.last_process_count.process.instance }}
{{ $values.last_process_count.Value }}
{{ $values.last_process_count.server.instance }}
{{ $values }}
{{ $values.server_health_condition.Value }}
{{ $values }}
{{ $values.server_health_condition.value }}
{{ $values.last_process_count.value }}
{{ .Values }}
{{ .ValueString }}
{{ $value }}
The values don’t populate as expected in AlertValues annotation that I manually created AND in the Description field. I can’t seem to get the values populated in the Alert Rules UI page AND the actual alert email I receive.
Here are the labels generated when I click on Preview Alerts
[ var=‘last_process_count’ labels={process=process1, server=SANDBOX} value=7 ], [ var=‘server_health_condition’ labels={process=process1, server=SANDBOX} value=0 ]
[ var=‘last_process_count’ labels={process=process2, server=SANDBOX} value=7 ], [ var=‘server_health_condition’ labels={process=process2, server=SANDBOX} value=0 ]
last_process_count - Reduce expression
server_health_condition - Math expression