Alloy locks the file and prevents it from being modified by other programs!

Grafana Alloy
1

Hello,
The following configuration locks the processed_logs.json file and prevents other applications from adding anything to it:

loki.source.file "processed_logs" {
    targets = [
        { __path__ = "C:\\scripts\\processed_logs.json" },
    ]
    forward_to = [loki.write.default.receiver]
}

loki.write "default" {
    endpoint {
        url = "http://192.168.1.2:3100/loki/api/v1/push"
    }
    external_labels = {
        job = "windows-security",
        logsource = "windows-eventlog",
    }
}

What is the solution?

Thank you.

1 Like
2

Is this file managed by your python script?

3

Hello,
Thank you so much for your reply.
Yes, a PowerShell script.

4

The issue is most probably powershell, please share partial code how you open and write and close file

Also why is there only one file? That approach seems very problematic design approach

5

Hello,
Thanks again.
The PowerShell script is:
https://pastebin.mozilla.org/cJLVLAaC

This script extracts the information I need from Windows Event Logs and is always running as a service.

6

Add-Content locks the file for reading and writing. So there is contention with alloy. This is to be expected. it could be the other around as well some other time, powershell has locked the file and alloy fails.

either you wait until alloy is done or figure out a different way of pushing logs.

for example, read and write 100 lines to a log file in a working directory then when done move it to the folder alloy needs kind of thing

7

Hello,
Thanks again.

How do I know if Alloy has done its job? Can I use Fluentd instead of Alloy?

8

You keep trying via script until it can or do the better approach,
You don’t need to know when alloy is done because it will work with files powershell no longer has a lock on

2 folders

powershell_process

alloy_process

Once ps finishes the file it has processed it moves it to alloy_process

Kind of approach,it is up to you how to design it

Or you could also go straight powershell to Loki a

10

Hello,
Thank you so much for your reply.
1- Can I use Fluentd to send Windows Event Logs directly to Loki?

2- By copy-based, do you mean that we have two processed_logs.json files? One of these files is used by PowerShell to generate logs, and the other is read by alloy. After PowerShell generates the logs in its own JSON file, it then replaces the file with the JSON file read by Alloy. Is that correct? If so, then the second file is locked by Alloy and cannot be replaced!

3- I don’t know anything about read-only access or shared file access.

11

Hello,
Thanks again.
Can PowerShell send data from a file directly to Loki?

12

Why even use a file? Just use your current powershell to send the JSON data directly to Loki

By the way I was able to use xpath to query 2 event IDs from windows event

1 Like
13

Hi,
Thanks again.
Great. Can you share the query?

14 
  eventlog_name          = "Security"
  xpath_query            = "*[System/EventID=5061] or *[System/EventID=5379]"
  poll_interval          = "0s"
  use_incoming_timestamp = true

in my case I get both 5379 and 5061

image
image461×544 33.1 KB

15

Hi,
Thanks again.
Let me explain further:
I want to know which file or folder was deleted by whom.The problem is that there is no file or folder name in ID 4660 and I need to extract the file or folder name from ID 4663.
When correlating events to determine which file or folder was deleted and by whom, the key is to use the unique identifier that’s common between both event IDs: the Handle ID.
In the Windows security event logs, when an object (like a file or folder) is accessed, a handle is generated and used throughout the process. Both event ID 4663 (which logs the access attempt along with details such as the object name) and event ID 4660 (which logs the deletion event but omits the file/folder name) include this Handle ID. By matching the Handle ID from event 4660 with the corresponding Handle ID in event 4663, you can reliably link the deletion action to the specific file or folder whose name is captured in 4663.

I need to write a query that compares the Handle ID of two IDs and if they are the same, then extracts the desired data.

16

that is a totally new requirement. please create a new post.

17

Hello,
Thanks again.
I have already done this.

1 Like