Alert based on percentage change of aggregated, weekly averages

Grafana Alerting
1

Hi Grafana fam, Quick question regarding alert configuration.

We have configured alerts to monitor specific metric and if they are over the threshold. (In our case, response Time of APIs.) However, is it possible to monitor the % change in past 7 days? Our goal is to set alerts to monitor the change % of average response time in the past 7 days. So basically an alert like this,

WHEN percentage_diff between A and B, when B is query A but with timeShift 7 days.

Due the the fact that alert is only supported in Graph panel in time series, not sure if it’s doable as query A always returns one single metric at a given time, instead of a daily/weekly/monthly average in an aggregated fashion.

Thanks all!

2

ok, what is the question?

3

Depending on your metrics provider. If it’s Prometheus, you can do something like my_metric / my_metric offset 1w. It’s current value of metric divided by the value of the same metric 1w ago. You should be able to turn this into alerting easily.

5

I want to set alert to compare with average from now to 7 days ago (A), and 7 days ago to 14 days ago (B). Then notify me if the change % between A and B is greater than 10%.

My b could’ve been more clear on my goal. Thanks for the help!

6

In that case,

my_metric / avg_over_time(my_metric[1w])

and

my_metric / avg_over_time(my_metric[1w] offset 1w)

should do the trick.

1 Like