Grafana alert does not route to >1 notification policy

Grafana Alerting
1

Hello all,

It appears to me on Grafana v10.3.3 the Continue matching subsequent sibling nodes feature doesn’t work on alerts. I’m attempting to have 1 alert send a notification to 2 separate contact points following the documentation to no avail.

My alert and preview routing:

image
image1224×991 123 KB

My Notification Policies:

image
image1013×215 15.9 KB

As you can see, my alert only routes to #test_contact_point as per the child policy destination = slack. I have also tried using 2 notification policies that aren’t in a parent-child relationship but that also doesn’t work. I cannot seem to have Grafana route to more than 1 contact point/alert.

Could someone advise on whether this is expected behavior for this version or a bug, or if I’m approaching this wrong?

2

Hi @desolar1um

I think the alert should be routed according to the child policy (destination=slack), as Grafana will select the most specific matching policy. Child policies take precedence over parent policies when both match.

In your case I think you need both policies to be of the same level in the policy tree. So, both should be at the top level (under the default notification policy).

Let me know if it helps

3

Hi Antonio,

Thanks for your reply, unfortunately I have already tried your suggestion to no avail.

4

if your alert has two labels label = 1, and other-label = 2, and you want to send notifications to different contact points. You can either:

  • Create two sibling policies (sibling = none of them are child policies. They are at the top of the policy tree).
    • Policy 1 with label = 1 – routes to contact point 1
    • Policy 2 with other-label = 2 – routes to contact point 2

Or

  • Create two alert rules. Each rule has one label to match one of the policies.

If none of that works, can you share a screenshot where we can see your notification policies? Redact any sensitive information

1 Like
5

This solved it for me, thanks @antonio

1 Like