I am running a self-hosted instance of Grafana 8.4.4 on Ubuntu 18.04 that I use along with InfluxDB to track a number of key metrics in my home lab and environment. As part of my setup, I built an alerting system that relies on Grafana json webhooks alerts, and this has been running perfectly for multiple years. (e.g. pre-dates 8.x) As you would imagine, I am still on the legacy alerting model.
My alerting solution parses Grafana webhooks json and then sends alerts via other avenue like text message and Pushover. As part of the parsing, it pulls out the current metric value and the alert threshold. (The alert threshold is pulled via a manual tag because I could not figure out how to get that data programmatically.)
In 8.3.4, the Grafana provided the webhooks json below, and note how “evalMatches” contains the current temperature and other metadata.
03-02-22 20:45 - {'title': '[Alerting] Upstairs Freezer alert', 'ruleId': 4, 'ruleName': 'Upstairs Freezer alert', 'state': 'alerting', 'evalMatches': [{'value': 20.1, 'metric': 'Upstairs Freezer', 'tags': None}], 'orgId': 1, 'dashboardId': 13, 'panelId': 5, 'tags': {'alertVal': '18'}, 'ruleUrl': 'http://<IP>/d/_fr6pntWz/home-appliances?tab=alert&viewPanel=5&orgId=1', 'message': 'Upstairs Freezer'}
In 8.4.4, “evalMatches” is empty as illustrated below.
03-06-22 10:45 - {'title': '[Alerting] Upstairs Freezer alert', 'ruleId': 4, 'ruleName': 'Upstairs Freezer alert', 'state': 'alerting', 'evalMatches': [], 'orgId': 1, 'dashboardId': 13, 'panelId': 5, 'tags': {'alertVal': '18'}, 'ruleUrl': 'http://<IP>/d/_fr6pntWz/home-appliances?tab=alert&viewPanel=5&orgId=1', 'message': 'Upstairs Freezer'}
I am trying to understand this change and if it is possible to get evalMatches populated in 8.4.4 legacy alerting. Is there a setting somewhere that I missed? Or is this perhaps a reason to move to the new alerting system?
Thank you in advance for any thoughts.