Report specific log entries via webhook

[teunvink1]

I’m trying to use Loki to detect specific log messages in my collection of syslog data, and report these messages via a webhook if they match a specific text.

I have a hard time figuring out how to write a alert rule and a notification for this. I can detect if there are any of these specific these messages:

count_over_time({job="networking"} |= "abcdefghijklmnopqrst"[10s]) or vector(0)

But if I then dump all data via a webhook, the value is the number of messages found of course, not the exact message lines. If I don’t use the ‘count_over_time’ function, I can’t determine if any matching lines are found.

Due to my inexperience with loki I’m probably missing something, but I wasn’t able to figure it out reading the docs. Any help would be appreciated.

[jangaraj]

I would create field (for example with whole log line for simplicity) with suitable parser and group by that field.

For example I have JSON logs and I want to be notified about watched action and user who did that:

sum(count_over_time(
{kind="mylogs"}[1m]
| json 
  user="user.name",
  action,
  timestamp,
| action="watched-action"
)) by (timestamp, user, action)

Then alert message can be (this example is for legacy alerting, new unified alerting has different syntax):

${timestamp}: ${user} ${action}