Set alert with percentage increase of a sum query

Grafana InfluxDB
1

Hi, I am trying to set an alert for a common (I believe) use case but I can’t seem to figure out.
Thanks for the help in advance.

Situation

  1. Data coming in every minute
  2. The data is number of transactions
  3. Datastore is InfluxDB

Alerting Idea
Number of transaction increased 100% in the last hour.

The following 2 kinds of windowing algorithm are both fine to me.
Algo 1 (fixed window)
Suppose there are total 100 transactions during 1:00 - 1:59
When there are >= 200 transactions at any time during 2:00 - 2:59.
I get alerted.

Algo 2 (sliding window)
Suppose it’s now 2:30pm
If number of transaction from 1:30pm to 2:30pm is 100% more than the number of transactions during 12:30 than 1:30pm
I get alerted

Not sure if “fixed/sliding window” are the correct terminology so please feel free to correct me.
I am also open to other suggestions.

What I tried
Query

SELECT sum(column) FROM database 
WHERE $timeFilter and <some condition>
group by time(1h)

Alert Configuration

Screenshot 2022-02-21 at 3.47.21 PM
Screenshot 2022-02-21 at 3.47.21 PM1339×442 26.8 KB

If I understand the documentation of influxdb and grafana correctly, this is not correct.
Let say current time is 1:30pm.
query(B, 1h, now) will return 2 data points.
12:00 | <number of transaction during 12:30 - 12:59>
13:00 | <number of transaction during 13:00 - 13:30>

Then based on this link, it simply compare the 2 data points which is not what I want.

So how do I achieve what I want? Thank you very much!

2

Hi there, did you ever get this working? I’m running into the same thing. I’d like to set an alert to fire if a node increases by say 20% within a few hours.

3

Hi @johnychevyeg

I think this thread should help.

Also, you did not specify your datasource, but some languages (PromQL, Flux, etc.) offer very good libraries for creating complex functions that may help as well (i.e. looking at the rate of change of a value, rather than the value itself).