I am making a dashboard that counts log messages and groups them by type and day.
I managed to get the correct logs and visualize them with this query:
sum by (MessageTemplate)(count_over_time({...} | json | MessageTemplate="foo" or MessageTemplate="bar" [$__auto] )) or vector(0)
HOWEVER… The days for that the logs are grouped under are shifted one day forward…
Here is a portion of the logs that renders the above dashboard, the ones I have marked should in my mind end up on the 15th of march, but in the panel they end up on the 16th.
Does this have to do with how the step (which is configured to [1d]) is configured? And how should it be configured otherwise?
Or is it something with how I represent the
The extra hour that gets added has to do with the timezone of my browser. The issue is the same when I’m running the dashboard in UTC.
Anyone that knows if it is fixable or what I am doing wrong?
I would like to avoid the ugly hack of subtracting 24 hours of each grouping in the result…
Just because your logs have Timestamp in them doesn’t mean Loki automatically reads them. When injecting logs into Loki the timestamp is the time of injection, unless you configure your log pipeline to adjust timestamp.
You should be able to find the real timestamp of your logs, which wouldn’t be part of your logs.
Your logs look good, your table view also looks good, that leaves the most likely problem to be your graph. Try creating a brand new visualization, slap your query on it and nothing else, and see what it looks like.
If that looks good, great. If not, double check your API call produced by Grafana and ran it by hand and compare results.
I’d also double check your interval properties. Also try not setting step. And check what transformation you have.
