When I am sending Grafana alerts to PagerDuty, multiple alerts grouped into single payload which leads to create single alert/incident in PagerDuty than individual incident.
How to generate individual payload for each alert in Grafana ? any pointers please ?
Hi @vjsre
Please do publish duplicate topics on the forum. I left a note on the other thread: I think that perhaps you should report this as a bug in the grafana repo.
Thanks @mattabrams, we found a work around to generate unique Dedup_key for every alert.
under notification policies, created a new policy with:
With this approach, PagerDuty is receiving each alert in individual payload and incidents are getting created.
Note: when you enable “override grouping”, it will ask you select something from the drop down listed below and it will be blank. don’t get panic. you just type the name there, it will accept it.
great job getting unblocked on this!
@vjsre Thank you for this solution!! I tried to do this but it didn’t work for me completely.
This is my Notification policy:
Alert Rule:
In my master PagerDuty channel, its still coming as 1 payload for 2 alerts instead of 2 separate payloads/alerts. What do you think I am missing?
@mattabrams @vjsre , Would you mind expanding more to this solution - as it seems the moment alert is sent to PagerDuty it get’s merged with parent alert and hence the user is not notified of any new alerts which are triggered in an event there’s an existing situation that’s in Triggered status
@vjsre, I’ve implemented exactly as per your suggestion however it still does not help with the cause.
Is there any specific settings which we are required to do on PagerDuty side? or should it be kept as Intelligent under Alert Grouping?
Hope to hear from you folks soon.
in Notification Policy settings enable group by host, payload will be splitted by hostnames
