I’m looking for help with Grafana alerts, specifically the “No Data” > Alerting scenario. Is there any way to make it more forgiving, e.g. allow for a couple failures before alerting?
Even if i reduce the ‘evaluate every…’ frequency it doesnt seem to matter; if the check occurs right on a ‘no data’ event it fires an alert regardless. I would like to allow for up to 30 minutes of ‘no data’. Anyone know how to do this, or can help steer me in the right direction?
The use case is we have a metric logger that logs in bursts, so it’s fine if we haven’t heard from it for 10 minutes or so. But if more than 30 minutes goes by without any activity, then that’s what we consider to be an alert condition.