Notification Policy based on Alert Status

Grafana Alerting
1

  • What Grafana version? 12 Enterprise

  • What are you trying to achieve? I’m trying to route Firing and Resolved alarms to a different contact point.

  • How are you trying to achieve it? By using Notification Policies and .Status of Alarm

  • What happened? No matter the status of alarm it always routes to the same contact point.

  • What did you expect to happen? I would like the status of the alarm be available when notification policies are being executed.

Thank you in advance for any help or suggestions.

2

This is tricky because notification policies route alerts based on alert instance labels.

And no label represents the alert status (firing or resolved).

This is by design because the label set is the identifier of the alert instance. If a label changed to represent the state, it would actually create a different alert instance.

I suggest using a webhook contact point and implement the routing logic in your service based on the .Alerts[].Status field in the webhook payload.

A distinct workaround could be:

  • Create two contact points. In one of them, enable Disable resolved message.
  • Create two sibling notification policies that match the same alerts.
  • Enable Continue matching subsequent sibling nodes so both policies process the alert.
  • Route each policy to a different contact point.

With this setup:

  • One contact point receives firing and resolved notifications.
  • The other receives only firing notifications.

Unsure if this is valid, perhaps, others can share alternative suggestions.

2 Likes
3

Thank you pepecano for the quick response and suggested solution. I didn’t think about the distinct workaround but it’s definitely a possibility.

thank you again