How to use Grafana Loki for Monthly API Usage (Long Term Graphs)

Grafana Loki
1

Cant seem to find out how to do this.

My company provides APIs as a Service and use an API Gateway.

In order to bill the customer for their usage, i need to keep a record of their Total No. Of API Calls on a monthly basis.

I can get this data from Loki and create a Graph in Grafana. But, Loki seems to be made for keeping data for a short period of time (probably 14 days on average?). After that, its best to delete it cause the loki storage would go into terabytes.

Im find with deleting the logs, but what i need most is a historical graph of the usage.

Meaning, i dont need the logs itself, but i need to be able to keep the counts (total number of api calls per month) and express it in a graph.

Ideally, id like to be able to keep the graph as long as 5 years so i can see the trend of the API Calls over the years.

How do i do this with Loki and Grafana?

2

There are a couple of options I can think of:

  1. If you have a prometheus installation already, you might consider using Loki ruler with recording rules to calculate a daily average / min / max / sum and write them into a prometheus store. See Alerting and recording rules | Grafana Loki documentation.

  2. You could do something similar as #1, but instead of ruler write your own script, and instead of prometheus write the data back into Loki. You’ll however need to configure a special retention policy for this data that you want to keep long term.

3

Hi Tony, thanks for the info!

We arent using Prometheus now. But we are using Zabbix with TimescaleDB.

Could we use that instead?

4

I don’t have experience with either of those. You might consider writing your own script in that case.