Calculation within aggregation function

On Grafana Cloud, the following expression evaluates fine:

max_over_time(series_xy{label_a="A"} [1h])

but this one breaks:

max_over_time((series_xy{label_a="A"}) [1h])

What I was trying to do is perform a calculation and then max_over_time the result. This is apparently not working. Why? The calculation inside max_over_time should go like “current value by its average value over 1h”

Hi,

When I tried to reproduce it, I got bad_data: invalid parameter "query": 1:52: parse error: ranges only allowed for vector selectors and I’ll assume you got the same error.

TL;DR

In your case you need to use a subquery, since the series selector in parentheses is not a series selector anymore - it’s a series selector in parentheses (at least that’s how I understand it :smile:).

Longer explanation

The error says the ranges are only for vector selectors (captain obvious ik, but I’ll explain that). It means that your range selector (time in brackets - [1h]) might only be after the} sign like in your first example. But when you wrap your selector into a set of parentheses, you no longer can put [1h] after the } sign. How I understand it is that prometheus will first evaluate the expression inside parentheses, then return it as an instant vector, since no time in brackets were provided. Then it cannot give you the data from last hour because it doesn’t know what resolution you need. That’s why you need to use a subquery - to tell prometheus that this one should return data from one last hour in e.g. 1m resolution - [1h:1m].

Subqueries - how they work

If you already know, you can skip this.
I’ve once seen a fine analogy and I’ll use it here. For example, we’ll use a metric pokemon_caught_total. We want to calculate the max_over_time over last five minutes with one minute resolution from query sum(increase(pkemon_caught_total[5m])). Now how would you do that without max_over_time function? You could type the expression into Grafana, select last 5 minutes and set min step to 1m. Now you can see, that the max from this setting is 107 (screen below - I have set time in stone, so the results will be predictable):

Since you had to tell Grafana the time range (5m) and resolution of the data (1m), you should tell prometheus the same, it’s only fair. So we’ll have the query:
max_over_time(sum(increase(pokemon_caught_total{}[5m]))[5m:1m]),
where
max_over_time(<query we already had>[<time from time picker>:<data resolution from "min step">]) which will produce the following chart:

And, as you can see, in 19:15:00 the value is 107 and it’s being kep. Now if I take the last 15 minutes (I increased resolution to 15 seconds):

The value 107 is being kept for 5 minutes (up to 19:20), then it drops to 101. Plot without max_over_time for the “current” last 5 minutes:

The point in 19:15 is not being taken into consideration anymore.

Hope that helps!