Grafana 8 Alerting Value Template

[masterblaster]

Hey Guys,

We did some testing with the ngalerting.
Setup:
Grafana 8.1.2
Datasource is elastic.

I have some Questions now:

1. In the alert message, i want to have the url to the dasboard as HTML. I tried the following:

{{ define “alert” }}
{{ range .Alerts.Firing }}
[New Alert]
{{ range .Labels.SortedPairs }}
{{ .Name | title }}: {{ .Value | title }}
{{ end }}
Silence alert: {{ .SilenceURL | safeHtml }}

Dashboard: {{ .DashboardURL | safeHtml }}
{{ range .Annotations.SortedPairs }}
{{ .Name }}: {{ .Value }}
{{ end }}
{{ end }}
{{ end }}

Did i missed something?

2. I want to have a nice and clean alert. But i can not format the value somehow:
   
   
   
   I just want the Lable and the Value. Not the hole string.
   How can i format that?

[mattabrams]

Hi @masterblaster

What kind of alerting rule did you create: a classic condition or a multidimensional rule? I believe message template only works with multi-dimensional rules. Check out this link for an explanation of the two and a brief intro to alert templating:

Create Grafana managed alert rule

[masterblaster]

Hey @mattabrams,

I think i createt a multi-dimensional rule. Because the template is working, just not in that way i wanted.

[mattabrams]

According to your screenshot this is using a classic condition rule:

Maybe try reformulating the alert to use reduce or something and see how that impacts your templating?

[janczewskimarcin]

hello, did you managed to solve your problem?

I have the pretty same issue.
I was able to configure multi rule:

but having a problem with value format.

{{ define “message-sst” }}
{{ range .Alerts.Firing }}
{{ range .Labels.SortedPairs}} {{ .Name}} : {{ .Value }}, {{ end }}
{{ end }}
{{ end }}

I need to extract only one exact label name and value for every entity in the new line or at least after the comma…

how to do it? I suppose I should use one of these:

Template data | Grafana Labs

[masterblaster]

Hey, Not yet

I tried today with the math expression an not the classic.
But there i got the follwing error:
invalid format of evaluation results for the alert definition : frame cannot uniquely be identified by its labels: has duplicate results with labels {}

No idea how to fix this.

[janczewskimarcin]

hej,

I managed to work around the problem by using a variable in the configuration for the alert.

then in message template

Right now have other issue
How to disable pending state or how to work around it to be able not to wait for another 12h to execute check for pending condition

[masterblaster]

Hey,

So it worked with the reduce in between, but there i miss the last value, because we cannot work with average.

The problem ist that my info is in a single row and not in different rows like yours.

I put that in on our end:

So its just two minutes till the alarm gets fired in my understanding.

[janczewskimarcin]

1. use “SUM” it will do the same as “Last” if you set a proper time rande for your query.
2. depend on your Garafan version set “evaluate every” on the 30s if don’t you will not receive a notification when the alert starts to firing. (its a bug that will be fixed in 8.1.7)
3. you will not able to set pending on zero. there should be such an option …
4. it doesnt matter how many tag’s you will have at the end. One or more …

[ged]

Hi guys,
Using Grafana 8.3.0 with Prometheus datasource.
I’m testing local disk usage alert having it configured as shown below (using query, math expression and reduce expression):

Alert is sent via email, and what I receive in email’s body is:

And here is the question:
In Value section I can see value=1 which I believe is result of expression $Local_disk_usage > 80 however exact value of disk usage in this case is 84.
Is it possible to extract this 84 and show in message body with message template ?

Thank you in advance !
Ged

[grimzly]

This is the query setup I use

Then for ‘Alert Detail’ I use the following in which you’ll have to use you’ll have to translate to your own label names:

{{ $labels.exported_instance }} on {{ $labels.instance }} ({{ $labels.job }}) disk usage at {{ $values.top_value.Value | printf “%.2f”}}%