Grafana Alloy | Filter log lines and transforming to JSON

Grafana Alloy
1

Hi Community,

We have a use case to filter a specific log from all log messages which like which looks like this “Critical Message – [Service:ServiceName, Title:Service Started, ErrorCode:CH0000, Severity:INFO, Date:2024-07-19-12-36+0000] Service Version Service Instance restarted” and transform it to json to extact values of key like “Service”,“Title” etc.

i am trying to solve using loki.process, but no luck as of now.
loki.process “alert_logs” {
forward_to = null
stage.pack {
labels = [“filename”,“hostname”,“type”]
}
stage.json {
expressions = {logline = “_entry”}
}

    stage.json {
      source      = "logline"
      expressions = {message = ""}
    }
    stage.labels {
      values = { "message" = "" }
    }
    stage.match{
      selector  = "{message=\"Critical Message --\"}"
      action = "keep"
      drop_counter_reason = "non-alert log"
      stage.output {
          source = "message"
      }
    }
   stage.regex {
       expression = `Critical Message -- \[Service:ServiceName, Title:(?P<title>.*), ErrorCode:(?P<errorCode>.*), Severity:(?P<severity>.*), Date:(?P<datetime>.*?)\] (?P<msg>.*)`
    }
}

Also is there any way to debug loki.process stages?

1 Like
2

Your log is not in JSON format, so you cannot use json filter here. Just use regex with group capture.

3

Thanks for suggesting. I am looking for logs filtering solution where I can keep only selected logs and drop all remaining. I have checked sub component under loki.process, stage.drop take expression to filter logs to drop, I guess it do not support negate expression to drop all other. Second option may be stage.match but still I see it passes all log line to forward list of Loki.process. can you suggest what to use in this case or am I missing some bacis concept.

4

Can you provide some example log and what exactly you are trying to achieve?

One more thing to keep in mind is that the regex filter is RE2 under Golang (see Syntax · google/re2 Wiki · GitHub), with its own caveats.

5

Hi @tonyswumac
We have some of the Kubernetes pod logs in a plain-text format. Something like this:

ts=2024-12-09T14:28:38.15226413Z level=info msg="Re-opening moved/deleted file /var/log/pods/grafana-alloy-infra_grafana-alloy-infra-4p6p5_155c78eb-febb-4b3c-ab28-ea366696b18c/alloy/0.log ..." component_path=/ component_id=loki.source.file.logs_config_kubernetes_pods

To use the JSON filter, the logs are initially supposed to be in JSON format, right?
Can Alloy somehow transform plain-text logs into JSON format before sending them to Loki?

6

I don’t believe this is possible.

1 Like
7

I believe Promtail supports this feature of transforming plain text to JSON format , but as Promtail is deprecated what is the solution here ?

We are moving out of Promtail and using alloy instead
I have plain text file & I want to convert it into Json format before sending to loki using alloy

please suggest

8

please post sample log lines from that text file

9

DateTime: 2025-03-03T04:49:12.6074358+00:00 | FunctionsStructure: General: Health | Signals: Health | Value: warning | Quality: good (process) | PlaceHolders: None
DateTime: 2025-03-03T04:49:16.6093329+00:00 | FunctionsStructure: General: Protection on | Signals: Protection on | Value: True | Quality: good (process) | PlaceHolders: None

A sample config.alloy That I was using for the same is as below -

logging {
level = “debug”
format = “logfmt”
}

loki.echo “alloy_log_scraper” { }

loki.write “logs_alloy” {
endpoint {
url = “http://loki:3100/loki/api/v1/push”
}
external_labels = {“cluster” = “my-cluster”}
}

local.file_match “logs_alloy_log_scraper” {
path_targets = [{
address = “localhost”,
path = “/somefolder/*.log”,
instance = constants.hostname,
job = “alloy_log_scraper”,
}]
}

loki.process “logs_alloy_log_scraper” {
forward_to = [loki.write.logs_alloy.receiver]

stage.regex {
expression = “DateTime: (?P[^|]+) \| FunctionsStructure: (?P[^|]+) \| Signals: (?P[^|]+) \| Value: (?P[^|]+) \| Quality: (?P[^|]+) \| PlaceHolders: (?P.*)”
}

stage.timestamp {
source = “DateTime”
format = “RFC3339Nano”
location = “UTC”
}

stage.json {
expressions = {
DateTime = “DateTime”,
FunctionsStructure = “FunctionsStructure”,
Signal = “Signal”,
Value = “Value”,
Quality = “Quality”,
PlaceHolders = “PlaceHolders”,
}
}

stage.labels {
values = {
DateTime = null,
FunctionsStructure = null,
instance = null,
}
}
}

loki.source.file “logs_alloy_log_scraper” {
targets = local.file_match.logs_alloy_log_scraper.targets
forward_to = [loki.process.logs_alloy_log_scraper.receiver]
}

10

Any suggestion how to achieve this @tonyswumac @yosiasz

11

I wouldn’t recommend you to change line format, especially into json. If you really try you can probably do it with template, but it’s not reliable, and there is no validation, and in my opinion you are just asking for trouble. If you really want to do this you should perhaps look for other log agents that support this, such as logstash (i think fluentd does too).

You have pattern and regexp filters on Loki for parsing arbitrary logs, what is the reason that you have to transform your logs into json?

1 Like