Help needed with Loki and LogQL

Hi,

Im having a issue with querying.
First let me explain how we set this up.
We’ve got a syslog-server thats is getting all the logs (syslog-ng) on a FreeBSD-server.
The logs are stored like this:
On the disk: /disk/syslog/$FULLHOST/$YEAR-$MONTH-$DAY.log
logs: (“$ISODATE $HOST ($FACILITY/$LEVEL) [$PROGRAM] $MSGONLY\n”)

Every host get their own folder and each day get a file for in that folder.
A line looks like this:
2023-04-04T10:11:53+00:00 HOST (local1/err) [Aug] 7 23:33:22 zzz.xxx.ccc.yyy TRAP:Pw 90 Jitter buffer underflow Put your string here

What I’m trying to achive is to count the top 50 hosts each day.
So for 2023-04-03 I want to see what 50 hosts have logged the most.

This is my test query:
topk(10, sum(count_over_time({job=“syslog”} | pattern “<> <message_type> <>” [1h])) by (host))

You are trying to sum by host, but your pattern name is message_type. Also, try peeling the onion layers one by one to see if you can figure out which part of the query failed.

1 Like

Thank you for the response.
I’ve got the sum(count_over_time) to work.
But when I start using topk(10…
It sometimes shows more then 10 hosts.

topk(10,
sum(count_over_time({job=“syslog”} | pattern “ <message_type> <_>” |~ “2023-.*”[1h]
))
by (host))

(the “2023-.*” is just because promtail is still reading old log files with wrong timestamp).

If you got time for one more question.
The [1h] vs time-range, whats the difference?