Grafana sending multiple emails for the same alert

Grafana Alerting
1

Hi All!

I’ve tried searching the forums for this topic but didn’t see anything resolved about this.
I have a problem where grafana(v9.5.5) is sending multiple emails for the same alert.

I have an alert rule that evaluates every minute for 10 minutes. At the end of the 10 minutes, it will fire. This alert gets processed by the notification policy.

The notification policy is currently set as such:
Group wait: 1m15s
Group interval: 1m15s
Repeat interval: 1d

The group by is set to [grafana_folder, alertname] and is correct.

The result of this is that i’ve received 3 duplicated emails. The Values section of the email is exactly the same for each email.

Is there any way to stop it from sending 3 duplicated emails? It should only send 1 email with all the alerts in it.

2

Hi! Are you using Grafana HA (High Availability) by any chance?

3

Is there any chance you have grouping disabled?

image
image713×105 7.91 KB

4

I have no idea, how should I check?

5

Nope! I have set my group by to grafana_folder and alertname

6

Are you running Grafana yourself or are your running a Managed Grafana provided as a service?

7

Well… i’m using my company’s grafana. I’m not too sure as to its status

8

It sounds like your company is running Grafana in HA but hasn’t set up gossiping (Enable alerting high availability | Grafana documentation). I would recommend checking with whoever manages your Grafana installation.