How do I prevent duplicate data from being sent?

Grafana Alloy
1

Hello,
I have a PowerShell script that stores Windows Event Logs in the processed_logs.json file. This script runs as a service and creates the processed_logs.json file each time. The Alloy configuration is as follows:

loki.source.file "processed_logs" {
    targets = [
        { __path__ = "C:\\scripts\\processed_logs.json" },
    ]
    forward_to = [loki.write.default.receiver]
}

loki.write "default" {
    endpoint {
        url = "http://192.168.1.2:3100/loki/api/v1/push"
    }
    external_labels = {
        job = "windows-security",
        logsource = "windows-eventlog",
    }
}

Because each time the file is created by the script, Alloy sends the whole file again, so duplicate reports are sent to the server. What solution do you suggest to solve this problem?

Thank you.

2

any reason you do not want to use loki.source.windowsevent

3

Hello,
Thank you so much for your reply.

1- Because of this. I was unable to write a Loki query that would compare the handle ID for different IDs and extract the required information if the handle ID was the same.

2- If instead of creating the file each time, I just append new data to the end of the file, then will Grafana Alloy only send new data?

1 Like