How to add a string to an alert

Grafana version 9.3.6

There is a dashboard with a panel in the form of logs.

Example:
192.168.122.33 - - [01/Feb/2023:10:51:56 +0000] “GET /test HTTP/1.1” 404 555 “-” “Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36” “-”

There is a need to add a log line or part of a line to grafana notifications.

Example:
Resolved

Value: [no value]
Labels:

  • alertname = Logs 404
  • grafana_folder=test
    -test=telegram
    Annotations:
  • description = “GET /test HTTP/1.1”

Is that possible?

@stanisslavcik if your logs have a field for message, you can include the message label in your alert template. This example is for a Loki alert.

You should be able to see the available labels by using the preview alerts button then hovering over the i in the info column for the alert:

When the alert is firing it will look like this:

I found another solution here if you don’t have a message field pre-defined. I’m working on incorporating this into a loki alerting tutorial :smiley:

In this example, my source log line looks like this and I am using a grafana managed alert

2023-04-22T02:49:32.562825+00:00 level=info method=GET url=/ status=200 duration=171ms

It’s important not to sum your aggregation in the query or you’ll lose the label data.

I used the pattern parser to capture part of the log line into a label called message. When selecting the portion of the line to capture, you’ll want to capture only the unique part of the line you want to use to create a series to alert on. Otherwise, you will end up with each line as its own series and be unable to aggregate in an expression.

So to compare the log line & parsing pattern:

Log line:2023-04-22T02:49:32.562825+00:00 level=info method=GET url=/ status=200 duration=171ms
Pattern syntax: pattern <> duration<>

The notation <_> in the pattern excludes the date from the captured pattern
Everything between the date and duration are captured into a new label called message.

Message label output: level=info method=GET url=/ status=500