Hi,
I configured pager duty webhook for alerting from a dashboard panel in grafana. However, when the alert has been met, the notification to pager duty doesn’t seemed to be generating and i couldn’t notice anything from state history.
Any suggestions on how this can be resolved or inspected with the alert notifications
Thank you.
If there is nothing in state history how do you know it has triggered ? Is the state changed ? Can you show your alert notification settings ?
Hi Torkel,
Thank you for replying back to me and i apologize for the delayed response. I have attached the screen shots of the alert configuration state history for the panel i have below. The alert rule i was trying to set up is if the cluster_health_value is >0 or < 0 i would like to fire an alert through pagerduty. The threshold we have set up for the alerts i.e >0 would signify there is an issue with the cluster and immediate attention is needed and if <0 then it is under some maintenance mode. with equal to 0 it is functioning as expected.
I have tried to setup the alert with conditions >0 and <0 and could never get an alert fired which i feel might be due to the condition not evaluated, however i tried to fire the alert with the condition equal to 0 and i haven’t observed anything in state history. kindly, let me know what i could be miss configuring thats preventing the alert to be triggered. I am using grafana-4.4.1 version on RHEL-7.
I have configured the pagerduty webhook in notification channel and was referring to that notifications tab of alert configuration.
Thank you.
I have updated the alert config to the following and noticed test rule to be true. But, i haven’t seen any notification triggered to pager duty.
Hi,
I was able to resolve this finally and the issue was with the following config property in grafana.ini said to be not true. Thank you
execute_alerts = false
