First, I have been trying for days to figure out what should be a very simple, straight-forward task - Create a human readable message of a firing alert. In my case the alerts are going to either email or MS-Teams. Grafana official documentation is un-clear and inadequate. The video posted does not touch on message formatting at all. (https://grafana.com/docs/grafana/latest/alerting/contact-points/message-templating/template-data/)
Just searching on this forum, I find MANY people frustrated and calling for help on this issue, and few if any responses from Grafana. There are some forum members suggesting other posts and posts at 3rd party locations. Expecting people to go elsewhere than Grafana.com for so basic a function as formatting messages is absurd. IF those posts are accurate, then their explanations need to be incorporated into Grafana official documentation, not left hidden away in the forums where users might or might not find them, or linked on a third party site.
Second, it appears that very significant coding is required to format alerts so that they provide concise, human readable messages, to whit:
{{ define "myalert" }}
[{{.Status}}] {{ .Labels.alertname }}
Labels:
{{ range .Labels.SortedPairs }}
{{ .Name }}: {{ .Value }}
{{ end }}
{{ if gt (len .Annotations) 0 }}
Annotations:
{{ range .Annotations.SortedPairs }}
{{ .Name }}: {{ .Value }}
{{ end }}
{{ end }}
{{ if gt (len .SilenceURL ) 0 }}
Silence alert: {{ .SilenceURL }}
{{ end }}
{{ if gt (len .DashboardURL ) 0 }}
Go to dashboard: {{ .DashboardURL }}
{{ end }}
{{ end }}
Step 2: Configure a template to render entire notification message.
{{ define "mymessage" }}
{{ if gt (len .Alerts.Firing) 0 }}
{{ len .Alerts.Firing }} firing:
{{ range .Alerts.Firing }} {{ template "myalert" .}} {{ end }}
{{ end }}
{{ if gt (len .Alerts.Resolved) 0 }}
{{ len .Alerts.Resolved }} resolved:
{{ range .Alerts.Resolved }} {{ template "myalert" .}} {{ end }}
{{ end }}
{{ end }}
Are you kidding – WTH Grafana?! No explanation of how this code was derived. No commenting of the code so that we can modify it for our individual needs. No explanation of syntax, or options that are possible. This is not an enterprise solution - this is a science experiment!
Additionally, the documentation does not explain WHERE to place this coding:
Do I place it in the Alert Rule (if so, which of the six options does one choose? No info, no hint provided)
OR maybe i past code into the Contact Point message?
I’ve wasted a whole lot of time with trial and error – is my code wrong, or am I putting it in the wrong place - it is anyone’s guess because as there is next to no documentation.
@praveenmayantha is absolutely correct – Alert Manager message formatting is a s**tshow!
Grafana - Please thoroughly document what you have in place now, and please strongly consider cutting out the ridiculous coding required to do something as simple as format a message.
.
