Including Graphite metrics in Grafana alert message?

This is probably really simple, but I’ve had no luck searching - how to include the metric value in an alert template? I have data going into Graphite and an alert set that if a value exceeds a threshold that an alert is sent. I’d like to send that value in the text of the alert message. My Graphite query is:

seriesByTag('name=freezer.temperature','unit=1')

I get the alert message and would like to include the actual value in the message. I know there are variables available of the form ${Value} and I’ve tried to figure out what that syntax might be, like $(seriesByTag(‘name=freezer.temperature’,‘unit=1’)} and ${‘name=freezer.temperature’,‘unit=1’}. I’ve dug through a lot of the docs, but I’m probably missing something that is a total newb answer. What am I doing wrong here?

Hi @roba,

What notification channel are you using for your alert? Can you share some screenshots? The more info the better. I have an alert set up for the discord channel, and the query AND value are passed into the message by default. No variables configured:

Matt, thanks for your response. I had been trying to use the Page Duty notification channel - particularly I was looking for SMS notification. Page Duty send the contents of the alert message box to as the SMS body. I wanted to include the actual reading that triggered the alert.

So my query looks like this:

and the alert like so:

I just can’t figure out what to put for %{???} in the screenshot to include the value in the message body.

Hi @roba,

What version Grafana are you running? It looks like you’re sending this alert a webhook and pager duty. If you remove the webhook and just use the pagerduty notification channel, does the response change?

If you share the details of your notification channel config for pager duty I’ll match them and test this on my end

I added the webhook later. It doesn’t change the message sent to Pager Duty. The message still comes out with the text entered in the message box either way.

The webhook post json does contain the value of the reading at the time the alert was tripped and Pager Duty seems like overkill for sending an SMS message, so I’m thinking of consuming the webhook with IFTTT and sending a notification via PushBullet or something. I’m not sure at this point I need the value in the message body if I have it in the webhook. It might be nice to figure out how to include it, but I think I’m going to change my alerting. As far as the Pager Duty config, it was the default configuration for Pager Duty - there really isn’t much to customize there.

FYI Grafana 8 gets shipped next week with a lot of improvements to alerting and templating alert messages

This topic was automatically closed after 365 days. New replies are no longer allowed.