I’m trying to divide my alert rule template into a section when firing and a section for recovery:
My template looks like this:
{{ if eq $values.B.Value 1.0 }}
internal temperature is {{ $values.A.Value | humanize }} °C, exceeding the 30°C threshold.
{{ else }}
internal temperature is back below the 30°C threshold.
{{ end }}
where $values.A.Value contains the actual temperature from the data source and $values.B.Value is the threshold. Alerts are going to Slack.
However, I get recovered Slack messages with descriptions containing the firing message:
temperature is 30.3°C, exceeding the 30°C threshold.
I’m using .Status in the notification template, which correctly shows the Slack message as recovered:
{{ if eq .Status "firing" }}:fire: FIRING{{ else }}:white_check_mark: RESOLVED{{ end }} — {{ .CommonLabels.alertname }}
So it looks correct, except that the alert description is incorrect
So what am I doing wrong here? Is in incorrect to compare on $values.B.Value, should I compare on the temperature value instead, or should I do something competely different to achieve the correct recovered message?
So this is what my message looks like. Since it’s Resolved according to the header, .Status cannot be firing. But the description indicates that it should be firing according to my template, i.e. temp > 30 and $values.B.Value == 1.
Root cause → annotations ($values, $labels) render once, when the alert fires, and are never re templated on resolve , that’s why your description stays stuck at "30.1°C, exceeding..." even though the header correctly shows RESOLVED. Status isn’t available in the annotation context at all, it only exists in notification templates.
Fix → move the firing/resolved wording into the notification template, per alert (not against the top-level .Status, which is just an aggregate across the whole group
And simplify your alert rule’s description annotation to just state the raw value, no branching →
internal temperature is {{ $values.A.Value | humanize }} °C
Then in your notification template, don’t replace your existing header ,combine them. Keep your current header line as it is and add a range .Alerts block below it for the description →
{{ if eq .Status "firing" }}:fire: FIRING{{ else }}:white_check_mark: RESOLVED{{ end }} — {{ .CommonLabels.alertname }}
{{ range .Alerts }}
{{ if eq .Status "firing" }}
internal temperature is exceeding the 30°C threshold. {{ .Annotations.description }}
{{ else }}
internal temperature is back below the 30°C threshold.
{{ end }}
{{ end }}
More worked examples of combining range .Alerts, .Status, and .Annotations →
Thanks for your very detailed reply. I completely understand what you are saying. My only problem with this way is that I have all different kinds of atmospheric metrics, that are all using the same contact point. That’s why I wanted to avoid having words like temperature, 'humidity etc in the notification template. This way, I’ll have to create a contact point for each such metric. But maybe I’ll just have to rephrase my alert template and avoid the outer if-else instead.
I think the correct way in my case is to create a custom annotation that contains the resolution description. Then I can test of .Status in the notification template and show either description or resolution.
That’s exactly right, and it’s the cleaner design for your case → keep the notification template fully generic and let each alert rule carry its own pre written text.
Add two annotations to your alert rule (no if/else needed in either they’re just static strings, so they’re safe regardless of when annotations get evaluated):
description: internal temperature is {{ $values.A.Value | humanize }} °C, exceeding the {{ $values.B.Value }}°C threshold.
resolved_description: internal temperature is back below the threshold.
Then your notification template just picks between them based on .Status, without ever mentioning “temperature” or “humidity”:
{{ .CommonLabels.alertname }}
{{ range .Alerts }}
{{ if eq .Status "firing" }}{{ .Annotations.description }}{{ else }}{{ .Annotations.resolved_description }}{{ end }}
{{ end }}
Since .Annotations is just a key value map, .Annotations.resolved_description works the same as .Annotations.description no special syntax needed for a custom annotation name.
This gives you one contact point and one notification template for all your atmospheric metrics each alert rule just supplies its own description and resolved_description text, and the template only decides which one to show, based on the live .Status.
