I have most of my log messages in this format:
[12:25:13 INF] Topics subscribed to: ["remapped-predictions"]
The “detected_level” label is “unknown” for all of the messages, so I’m trying to use this approach to parse and change the log level string into a label:
loki.process "pod_logs" {
stage.regex {
expression = `^\[(?P<time>\d{2}:\d{2}:\d{2}) (?P<level>[A-Z]+)\] (?P<msg>.*)$`
}
stage.template {
source = "level"
template = `{{- $level := upper (default "" .Value) -}}{{ if eq $level "INF" }}info{{ else if eq $level "DBG" }}debug{{ else if eq $level "WRN" }}warn{{ else if eq $level "ERR" }}error{{ else if eq $level "FTL" }}fatal{{ else if eq $level "TRC" }}trace{{ else if eq $level "CRT" }}critical{{ else }}{{ end }}`
}
stage.labels {
values = {
"level" = "level",
}
}
stage.output {
source = "msg"
}
forward_to = [loki.write.default.receiver]
}
However, this does not work and the “level” label is missing on all logs.
And once i’ve got this working, how would I go about supporting multiple custom log formats?
You have too many keys named `level`, I would recommend you to try and distinguish them from each other somewhat. Also you probably need to break your template stage into two, first to assign upper value, second for the if/else logic.
Something like this (not tested):
loki.process "pod_logs" {
stage.regex {
expression = `^\[(?P<time>\d{2}:\d{2}:\d{2}) (?P<level_regex>[A-Z]+)\] (?P<msg>.*)$`
}
stage.template {
source = "level_upper"
template = "{{ .level_regex | ToUpper }}"
}
stage.template {
source = "level_parsed"
template = "{{ if eq .level_upper "INF" }}info{{ else if eq .level_upper "DBG" }}debug{{ else if eq .level_upper "WRN" }}warn{{ else if eq .level_upper "ERR" }}error{{ else if eq .level_upper "FTL" }}fatal{{ else if eq .level_upper "TRC" }}trace{{ else if eq .level_upper "CRT" }}critical{{ end }}`
}
stage.labels {
values = {
"level" = "level_parsed",
}
}
stage.output {
source = "msg"
}
forward_to = [loki.write.default.receiver]
}
If you have multiple custom format and you want parse them then you’d have to go through this process for each unique log pattern. This is why I generally prefer to keep my logs in JSON format when possible.
Thanks, unfortunately that didn’t do anything but I take your point about variable naming.
I can’t seem to get anything working that involves the regex stage. Even this won’t work:
loki.process "pod_logs" {
stage.regex {
expression = "^(?P<everything>.*)$"
labels_from_groups = true
}
stage.output {
source = "everything"
}
stage.static_labels {
values = {
test = "test_val",
}
}
forward_to = [loki.write.default.receiver]
}
I see the test label but I don’t see the “everything” label in grafana.
I can also see a lot of this in the logs:
ts=2025-08-12T15:50:04.525477222Z level=debug msg="extracted data did not contain output source" component_path=/ component_id=loki.process.pod_logs
Is there a more minimal use of regex i can use?
I tested it, seems to work. Here is my setup
Sample logs:
[12:25:13 INF] Topics subscribed to: ["remapped-predictions"]
[12:25:13 DBG] This is debug
[12:25:13 WRN] This is warning
[12:25:13 ERR] This is error
[12:25:13 FTL] This is fatal
[12:25:13 TRC] This is trace
[12:25:13 CRT] This is critical
Config in the process stage:
stage.regex {
expression = `^\[(?P<time>\d{2}:\d{2}:\d{2}) (?P<level_regex>[A-Z]+)\] (?P<msg>.*)$`
}
stage.template {
source = "level_upper"
template = "{{ .level_regex | ToUpper }}"
}
stage.template {
source = "level_parsed"
template = `{{ if eq .level_upper "INF" }}info{{ else if eq .level_upper "DBG" }}debug{{ else if eq .level_upper "WRN" }}warn{{ else if eq .level_upper "ERR" }}error{{ else if eq .level_upper "FTL" }}fatal{{ else if eq .level_upper "TRC" }}trace{{ else if eq .level_upper "CRT" }}critical{{ end }}`
}
stage.labels {
values = {
"level" = "level_parsed",
}
}
stage.output {
source = "msg"
}
Result:
Strangely this doesn’t work for me. My system didn’t recognise ToUpper function so i had to remove that step.
I changed the output of stage.template just to check that it was working at all:
stage.template {
source = "level_parsed"
template = `test`
}
And this worked and gave me: level = test in Grafana. So it seems that the regex step just doesn’t work. When looking at the live debugging output this is a sample of the input to this process:
[IN]: timestamp: 2025-08-13T10:08:35.518297043Z, entry: [10:08:30 INF] Connections to Neo4J will use: URL=neo4j://neo4j:7687, username=scheduledtransport_service
, labels: {container="scheduledtransportdb-event-predictions-alert", container_runtime="containerd", instance="test-tdm/scheduledtransportdb-event-predictions-alert-fake-default-xn7w9:scheduledtransportdb-event-predictions-alert", job="test-tdm/scheduledtransportdb-event-predictions-alert", namespace="test-tdm", pod="scheduledtransportdb-event-predictions-alert-fake-default-xn7w9"}, structured_metadata: {}
Is this what you’d expect? I tried adding .*? to the start of the regex in case it was operating on the whole string but without success. Will stage.regex be operating on the value of entry only?
It’s version v1.10.1 installed using the helm chart v1.2.1.
In the alloy logs:
ts=2025-08-15T07:56:30.592946093Z level=debug msg="extracted data did not contain output source" component_path=/ component_id=loki.process.pod_logs
I can’t be sure that the log corresponds with the screenshot but that’s pretty much all there is in the logs.
Try removing all processing and just let the log go through to Loki, see what it looks like. There is a chance your regex doesn’t fit your actual logs.
