TL;DR
I am trying to transform windows event logs in Alloy before writing the data to Loki in Grafana Cloud. The idea being to minimize the data sent and to provide the data in a format that is easy to work with in Grafana. I’m trying to drop unused data fields and extract data fields from nested objects.
Context
We have a Windows Server tool by SanDisk called GRAID that we use to setup RAID for NVMe drives. This tool doesn’t offer much in the form of metrics / logging. It does have a CLI, so what I’ve done is written a PowerShell script that logs the output of the CLI commands to the Windows Event log. I run the PowerShell scripts every 5 mins using Windows Task Scheduler to create a log.
In Windows Event Viewer, the data looks like this:
Alloy Pipeline
If the Windows Event data is uploaded to Loki without any processing, it looks like this in Grafana Cloud:
So given that, to transform the “raw” data, I implemented a loki.process
block as follows:
loki.process "graid_windows_logs_json_processor_1" {
forward_to = [loki.relabel.graid_windows_log_files_label_cleaner.receiver]
// stage.replace {
// expression = "<Data>|<\/Data>"
// replace = ""
// }
stage.json {
expressions = {
log_source = "source",
graid_event_type = "event_id",
timestamp = "timeCreated",
message = "event_data",
}
}
stage.labels {
values = {
log_source = "log_source",
graid_event_type = "graid_event_type",
}
}
// stage.json{
// expressions = {
// result.message = "message",
// result.graid_event_type = "graid_event_type",
// result.log_source = "log_source",
// result.timestamp = "timestamp",
// }
// }
// stage.output {
// source = "result"
// }
}
The parts that don’t work are commented out and are explained below:
Issue 1: Removing XML Data Tags
The first stage is there because I’m trying to remove the <Data>
and </Data>
tags from the "event_data"
JSON field. I thought to try a stage.replace.
This doesn’t seem to be working. It could have to do with how regex doesn’t work very well with XML. It could also just be incorrect syntax / implementation on my side.
Issue 2: Transforming the Output JSON Object
You’ll note that the stages in the loki.process
block that are uncommented are extracting data into labels. So, the final two stages (stage.json
and stage.output
) are to try transform the object that I send to Grafana Cloud into a final format like this:
{
"timestamp": "2024-07-26T09:55:01.8698452Z",
"errorCode": 0, //from event_data
"errorMessage": "" //from event_data
"result": [ {...}, {...} ] //from event_data
}
I thought to try map the data using the JMESPath syntax, but I think I’m getting it wrong. Also, if the stage.output
could accept a map(string)
it would make doing this a bit easier because then I’d just pass the new object fields in directly.
The lessons learned here can definitely be applied to any JSON object transformation with Alloy. Any help would be greatly appreciated!