Using regex to parse logs into labels with Alloy

Grafana Alloy
, ,
1

I’m sending pfSense logs to Alloy and trying to use regex to parse the logs and break them up into labels so I can organize the logs in Grafana. I’ve been at this for days and I’m close to the point of giving up. I’ve Googled, AI-ed, and tested the regex. Alloy starts and I get logs in Grafana, but none of my labels show up. Here’s my alloy config:

logging {
level = “info”
}

loki.write “default” {
endpoint {
url = “http://localhost:3100/loki/api/v1/push”
}
}

local.file_match “pfsense_logs” {
path_targets = [
{
path = “/var/log/pfsense.log”,
},
]
sync_period = “10s”
}

loki.source.file “pfsense_scrape” {
targets = local.file_match.pfsense_logs.targets
forward_to = [loki.process.pfsense_pipeline.receiver]
tail_from_end = false
}

loki.process “pfsense_pipeline” {
stage.regex {
expression = “^(?P\S+)\s+(?P\S+)\s+(?P.+?)(?:\[(?P\d+)\])?\s+(?P.+)$”
}

stage.labels {
values = {
ts = “”,
host = “”,
app = “”,
pid = “”,
}
}

stage.timestamp {
source = “ts”
format = “rfc3339”
}

forward_to = [loki.write.default.receiver]
}

Can someone please tell me what exactly I’m doing wrong here? Any help is much appreciated.

2

When using the regex stage, you need to either double escape special notation (for example, \\S) or use the character ` instead of " to enclose the expression string.

Also I would recommend not using values or ts as labels.

If you can provide a sample log string i can test your regex for you if you’d like.