Hi,
I have created a Grafana alert using Prometheus DB for HTTP response time when the value is greater than 0.05ms.
I am able to create the alert. But I also want it to display the exact value in the alert message. for ex; Response time is 0.06 which which exceeds the threshold of 0.05ms.
I have tried using this:
Response time for {{ index $labels “instance” }} is currently {{index $values “A” }} seconds, which exceeds the threshold of 0.05ms.
This doesn’t help. Can anyone fix this for me please?
Regards
Hi,
What do you mean by this doesn’t help? Where did you write the expression with templates? What is being sent in the message. Can you also screenshot the alert definition (in case of sensitive data you can cross it out).
This kind of templates (I think at least) should found its way in alert description or summary (the annotation and labels setting section).
Also… are you sure about the threshold? 0.05 ms sounds harsh for the person receiving the alert 
Hey @dawiddebowski, Thank you so much for the response. By “this”, I meant the annotation that I tried. But it did not give me the expected result.
About the threshold, please suggest me if I can do better.
Ok, the threshold is fine, the message might be a bit off - metric unit is second, not ms, so the threshold is 50ms, not 0.05ms
As for the annotation itself, it should work. After saving the alert, could you show the screenshot of the alert instance (wait for the first check, view alert and toggle one specific instance - the annotation should be templated into specific values)
In the screenshot, I can see B= some value. This is the value that I should be able to see in teh Alert message. where it says [no value]
The issue might (I’m not sure) be connected to the use of Classic Condition. What Grafana version are you using? In newer versions (8+), I’d suggest using Reduce and Threshold expression, since Classic Conditions get rid of the labels (hence the $labels.instance also returns no value). The $values.A.Value might be returning no value because it’s time series, not a single number. Since you’re using last function, you can change the query type from Range to Instant - you’d also query less data and Prometheus will thank you for that
To reiterate - Classic Condition (probably) makes it impossible to access instance label, while use of time series (probably) makes it impossible to access A value. In this case (if you don’t want to change anything) maybe using $values.B.Value would help (but instance label is still out of question imo)?
You can replace classic condition with Threshold (Add expression → Threshold). And in the threshold you can type is above 0.5 (then you can remove the classic condition). Can you also share the screen after the changes after clicking Preview button (to confirm which labels we have) - cross out sensitive data?
Time series data shows no data in preview.
It should be fine now (at least as far as I can tell ). You have the instance label in B expression, so Grafana should be able to add it to labels. I’m only a bit worried about Data is missing a time filed - does it also appear if you use code editor, not builder (imo it should show no data not, this message?).
Anyway, do you have problems with such defined rule?
Now we can see some data there.
Alright, so does it work now? Fun fact - today I’ve done similar thing (having Classic Condition and using tempating with instance) and I changed it to threshold expression and worked for me.
I’d reconsider using heatmap - AFAIK it does not return the same data. It should work with format TimeSeries and Threshold as long as you have type Instant query (threshold expects a single number value, query with type Range will return series of numbers which is not a single number). Alternatively, you can use Reduce between query and Threshold (it will allow you to reduce series of number to a single number using some reducing function, like Last or Total).
Hi Dawid,
Hope you have a great start to the new year!
I am working on SSL certificate expiry alert in Grafana.
I get the exact number of days that is left for expiry.
But I wanted it in description as annotation.
That is where I am currently stuck.
instead of {{$B}}, i want it to show the value as 371 (no decimal points)
Do you have a suggestion?
Hi, according to docs it should be {{ $values.B.Value }}
with {{ $values.B.Value }}, It gives description as: SSL certificate expiring in [no value]
I tried multiple options like {{$labels.instance}} and {{$value}}
But nothing works. Not able to find a way through this.
Is this the same alert or different? If different, can you screen the configuration?
It was a different alert that I was working on.
Good news…when I sent you the last text, I was working with classic condition.
Then meanwhile I changed it to Threshold and now I get the desired result.
Just one thing that I am still struggling with is the decimal point.
I want it in whole numbers because its (how many days is remaining)
Here is the snap for your reference.
Hi,
I fixed that issue.
I used ‘round’ in front of my Prometheus query and it rounded of the no. of days.
1 Like
