How to make Grafana/Prometheus summarize values by max or peak when zooming out?

Grafana Prometheus
1

Hello

I have a time series that plots a single measure from Prometheus. The query is basically measure_name.

My issue is that whenever the chart resolution goes down a step, as I increase the time interval, I lose all the peaks.

Here is the chart spanning 20 hours:

grafana_20h

Here is the same chart spanning 22 hours:

grafana_22h

As you can see, the chart becomes useless.

How can I make Prometheus return the maximum or peak value for each time step?

Right now it’s not even returning the average, maybe it takes the spot value at the beginning of the time interval, or something like that.

2

Try to make the Date Histogram - Interval is not Auto.

Put some value there, try to put 1m or 5m

Or the other way around,
Find the Query Option, an put Minimal Interval for 1m or 5m

Regards,
Fadjar Tandabawana

3

Where is that “Date Histogram - Interval” setting?

I tried setting Query Option, Minimal Interval to 1m, but it doesn’t change anything. Which is obvious, since it’s a min setting, not a max setting.

In any case, when I’m zooming out to, say, 1 week, I don’t want to download a data point for each 1m time step, that would be 10k data points.

What I’m trying to figure out is: when Grafana gets a data point from Prometheus, for example with a 10m time interval, I want it to be the peak value in that 10m span, not the first one or a random one. The average would also be cool, but peak is generally more important to me.

4

I found it!

The PromQL function to obtain the peak value for every interval, instead of the last recorded value, is this:

max_over_time(...[$__interval])

For instance:

sum(max_over_time(some_gauge_metric{option=~"$variable"}[$__interval]))
3 Likes