we are monitoring our infrastructure using telegraf, InfluxDB and grafana. We also use the alerts from grafana. One graph that we also use for alerts shows the memory usage of our systems grouped by host using a query like this:
SELECT mean("used_percent") FROM "mem" WHERE $timeFilter GROUP BY time(1m), "host" fill(null)
When configuring the alert there is an option to trigger an alert when no data is received. The problem is that this alert would only be triggered when all hosts don’t send data any more.
Is there a possibility to configure the alert to trigger also when only one single host stops sending metrics other than using a single graph + alert per host ?