This is feedback to product managers at Grafana, to provide yet another reality check on the poor state of alert templating. Specifically the critical and obvious core product functionality of generating human readable alerts.
The byzantine approach that the Grafana product takes, is from my perspective (20 years in IT) still painful and broken.
As a company using Grafana, we have repeatedly attempted to setup readable alerts, which we receive via webhook (Slack alternative, specifically Zulip). Despite years of revisiting this work we have still to date not been able to achieve acceptable alerts. To date only one person in the organisation reads Grafana alerts because the defaults are so verbose, ugly, and unpleasant to mentally parse.
I’m a manager who has been working on this directly today, because technical staff have been unable to achieve improvements to our alert messages, and good alerting is such important functionality. I conclude from the last 90 minutes of work, and also from the very extensive feedback here on Discourse and from other users (including multiple multi-year threads on the subject), that alerting templating in Grafana is still not fit for purpose.
The objective of this post is therefore simply to provide another data point and reminder for the Grafana team, that this functionality is still not functional or acceptable to many users.
In our case, this weak point of the product, combined with one or two others which we would otherwise be prepared to tolerate, makes us again look for a Grafana replacement which understands the importance of UX.
I hope that’s helpful.