Scaling Grafana alerts dynamically

Grafana Alerting
1

Dear Grafana users,

We have more than 200 servers that we are monitoring with Grafana currently. We need to setup alerts to get noticed about issues. The goal is to have one dynamic alert that monitors the disk space. We managed to have this alert but the way it sends the email is not what we want.
Currently, if the rule gets fired by 10 instances at the same time, it just sends one email containing the information about those 10 fired instances.
The ideal case is to send 10 separate emails with the specific message for each instance.

Would you please share your solutions?

Thanks a lot

2

@sinaps95 welcome to the :grafana: community!

It sounds like the alerts are grouped. I’d suggest to look at the notification policy that is sending the alert to modify the group settings. If you want to disable grouping, you can do that in the notification policy too.

https://grafana.com/docs/grafana/latest/alerting/manage-notifications/create-notification-policy/#grouping

1 Like