How to verify from the UI that alert has been sent

Grafana Alerting
1

Hi,
I am using Grafana 10.3.3

I have created one alert from the Grafana GUI using the label : severity = major

I have created one Webhook based contact point with the proper URL.

I have also created a notification policy and associated the alert with the contact point.

How do I check from the Grafana GUI …when the alert is fired , the notification is actually sent from Grafana ?

Please let me know.

2

Hi, you can either check logs for some message (I think it’s logged). Or you could look to the contact point page, under the name of your contact point there should be information about last delivery attempt and how much time it took. If the contact point wasn’t used, there should be “no delivery attempts” message or something like this

3

I have “test1” as the contact point …I could see the following in the Grafana Contact Page GUI

image
image1439×192 16.5 KB

Last delivery attempt 43 minutes ago …took 3ms ..

Does it mean the Contact point is properly picked up by the Alert ? Also , my alert is getting fired every 5 minutes …Then why the delivery attempt shows 43 mins ago ?

Any log on the Grafana side I can check ..

Thanks,
Nilanjan

4

How do you know your alert is fired every 5 minutes?
There should be some logs on evaluating alert rule for sure and I think for sending too but I’m not sure.

5

Hi, there are a few related topics here.

First, Grafana Alerting separates alert evaluation (the alert generator) from alert notifications (the receiver, also known as Alertmanager). You can read more about the architecture here:
https://grafana.com/docs/grafana/latest/alerting/fundamentals/notifications/#architecture

From the UI, you have some options to troubleshoot:

In addition, Grafana exposes performance metrics and logs that can help with troubleshooting, which differs depending on the Grafana edition:

Also , my alert is getting fired every 5 minutes …Then why the delivery attempt shows 43 mins ago ?

This depends on how notification delivery is configured for repeated alerts.
The repeat interval is by default 4 hours. See here for details on notification grouping and repetition.

Hope this gives you the right pointers to verify and troubleshoot your alerts!

6

You can check this directly in Grafana GUI → Alerting → Notification Policies → Contact Points → Test. Fire your alert manually or use the “Preview alerts” option to confirm if the webhook notification is sent. For more details, you can click here.