What Grafana version and what operating system are you using? Cloud version & MacOS
What are you trying to achieve?
I’m retrieving data from a treafik counter metric, called “traefik_service_requests_total”. This counter logs the requests that are being done to services, running on our VPS.
I’d like a barchart with multiple series, like so:
Service A (group):
A bar with total request count with status code: 200 (or 2xx)
A bar with total request count with status code: 500 (or 5xx)
A bar with total request count with status code: 400 (or 4xx)
Service B (group):
A bar with total request count with status code: 200 (or 2xx)
A bar with total request count with status code: 500 (or 5xx)
A bar with total request count with status code: 400 (or 4xx)
Service C (group):
A bar with total request count with status code: 200 (or 2xx)
A bar with total request count with status code: 500 (or 5xx)
A bar with total request count with status code: 400 (or 4xx)
We’re also not interested in the specific timestamps, but instead we would like an aggregation/sum of the requests in the last hour.
How are you trying to achieve it?
I’m trying to transform this data to the same shape as seen here: Grafana
But even when using transformations like “Labels to fields”, “Group by”, etc. I can’t achieve the same shape. I’m also quite new to Grafana, so I don’t know all the ins & outs yet.
What happened?
If you look at the image you can see the timeseries are divided and not aggregated. I’ve tried multiple ways to get the data in the same shape as here: Grafana
What did you expect to happen?
To have multiple “service” groups in a timechart, each with “status code” bars, that each show a sum of the requests from the last hour.
Reduce with mode Series to Rows - this will produce a single table with each time series reduced into one row (you most probably need Total calculation or use promql sum_over_time with instant query mode - this have some tweaks, I’ll describe later). Also toggle Labels to fields.
Group to matrix, in which you’ll set Column to service, Row to code and value to Total or something like that. This will produce the format you want.
Bar chart will produce bars next to each other in service groups. If you want, you can stack them (Bar chart => Stacking => Normal).
The tweaks I mentioned are related to the data and how prometheus handle them. Notice that in your case Grafana shows data in 15 seconds interval - I guess your query looks something like rate(traefik_service_requests_total[5m]) or increase(traefik_service_requests_total[5m]). If you’re using rate you should be alright, since the data is being averaged over the five minutes.
But if you were using increase, notice, that you have the increase over last five minutes. Imagine your data points look like [a1, a2, a3, a4, a5, a6] difference between a5 and a1. Now the next point is the difference between a6 and a2, so you calculated the increase of a3, a4, and a5 two times. Your data in that case might be misleading, probably only if you’re using increase though, which I doubt (most of the tutorials and Grafana show rate).
If you want some defaults, you can go Standard Options => Color scheme and pick Classic Palette (probably is selected by default). If you want to make custom colors per e.g. status code (which would probably make much more sense), you need to specify what color goes to what status code by fields override. You scroll to the bottom on the panel settings (on the right, where you selected bar chart) and there’s a button Add field override. There you can pick either Fields with name of Fields with name matching regex (will suit your case the best - I’ll post example with both). In there you can override the settings visible in the right panel for certain field:
As for the legend text, it probably depends which text you want to change - the status or the service name? Can you show an example of “I have this” “I want that”?
Also I noticed the status code “0” in the legends (bottom left of the chart). Is this something we can exclude?
Yeah, in PromQL you can filter out some labels with != operator, so your query would look like: rate(traefik_server_requests_total{job="coolify-proxy-primary-service", code!="0"}[5m])
So instead of “https-0-my-service-jfmcdniiuorwq@docker” I would like something like “My HTTP Service”.
I guess it’s not creatable from the incoming label, is it? I don’t think Grafana has anything like that, but you could use PromQL’s label_replacefunction.
In your case you could do something like this: label_replace(rate(traefik_server_requests_total{job="coolify-proxy-primary-service", code!="0"}[5m]), "service", "My HTTP Service", "service", "https-0-my-service-jfmcdniiuorwq@docker")
and so on for all the transformations you want to do (each transformation requires another label_replace calling) - that’s if it can’t be done by some regex expression (based on example it can’t but maybe it will be on the real data).
Was also wondering, are there any good “practical” courses for grafana out there?
Not that I’m aware of. Personally, I learned some basics at work, then I just had some problems I wanted to solve and I managed to find my way around. Then it was more and more “challenges” (of other people like in here) and that’s how I more-or-less learned You can try your luck on youtube or something like that, I don’t think Grafana has any official courses (there are some webinars I went to a couple of but I don’t know if you’ll find anything you’re looking for).
You can try your luck on youtube or something like that, I don’t think Grafana has any official courses (there are some webinars I went to a couple of but I don’t know if you’ll find anything you’re looking for).