Unexpected results with increase() function for Counter Metric in Grafana/Prometheus

Grafana Prometheus
1

  • What Grafana version and what operating system are you using?
    Grafana v10.1.0 (ff85ec33c5)

  • What are you trying to achieve?
    I am trying to calculate the total number of requests in a specific period of time.

  • How are you trying to achieve it?
    By using the increase() function over a time range.

  • What happened?
    I received results that do not match my expectations.

Description
Hello, I have a metric of type counter with the following labels:

  • host
  • method
  • handler
  • code

I am using this metric to calculate the total number of requests made by my application. Over the last hour, I performed 6 requests: 3 from host1 and 3 from host2.

When I use the following query:
increase(my_metric_counter[1h])

I get this result:
{code=“200”, handler=“myHandler”, host=“host1”, method=“GET”, path=“myPath”} = 2.00276
{code=“200”, handler=“myHandler”, host=“host2”, method=“GET”, path=“myPath”} = 1.00138

I was expecting to see 3 for each series instead.

What did I check?

  • I verified the metric values using a simple query (my_metric_counter), and the data shows 3 requests for each host.
  • I reviewed the Prometheus documentation and other forum topics but couldn’t understand why the increase() function returns these fractional results.

Can someone explain why this is happening and how I can adjust my query to get the correct total?

2

Hi,

Can you post how the series without increase invocation look on the last hour?

3

Hello, thanks for replying. When I run my_metric_counter it returns:
{code=“200”, handler=“myHandler”, host=“host1”, method=“GET”, path=“myPath”} = 3
{code=“200”, handler=“myHandler”, host=“host2”, method=“GET”, path=“myPath”} = 3

4

Could you share some screens from Grafana explore view from the last 3 hours? Of just the metric and the metric with increase invocation. Also, is the counter implementation in your application on the table (the code)? As far as I’m aware, increase should only calculate the difference between the two extreme points in the lookbehind window. Do you have any other points in the window? There’s also a possibility that the data is too low to calculate increase, as it’s just rate * number of seconds in the window (that’s why the results might have those decimals).
Also, do you remember how you were performing the requests? Was it all at one time, was it one request - 1 hour break - 2 requests?

1 Like