Alert rule goes into Firing but notification randonly not send

Grafana Alerting
, ,
1

i have an alert rule. In grafana I can see it correctly goes into a firing state. The issue is - sometimes the notification is not delivered. I thought it could be related to SMTP (I am using gmail) but my rules are configured to deliver to both, email and telegram. In this case both notifications are missing.

When I restarted Grafana the email eventually landed, but for next firing alerts it was missing again.

In logs I can see:
logger=ngalert.sender.router rule_uid=f10051c5-651c-4e59-946b-a1246becdfb9 org_id=1 t=2023-10-02T21:16:17.548377904Z level=info msg=“Sending alerts to local notifier” count=1

2

Hi! :wave: This sounds related to the grouping feature of Alertmanager and group timings. There is some good information about how it works here and here.

3

The documentation is for Prometheus alert manager, when I am using grafana’s default alert manager. My rule are based on InfluxDB queries.

4

Grafana uses Prometheus Alertmanager as the default Alertmanger :slight_smile: It’s embedded inside Grafana.

5

@michalveda @georgerobinson were you guys able to resolve this issue. I am also facing same kind of issue with grafana default alerting .