Actually Im trying to understand loki.process/relabel the logs coming from my application. My log format is being this way - 2025-02-14T09:17:58.280Z [info] - GET /api/featured/config { “method”: “GET”, “url”: “/api/featured/config” }.
Could anyone please let me know the log format of how it needs to be, so Alloy can process it properly…
Recently I changed it to this format → {“method”:“GET”,“url”:“/auth/logged_in”,“level”:“info”,“message”:“GET /auth/logged_in”,“timestamp”:“2025-02-21T07:16:48.362Z”}
I have used stage.json within the loki.process
I am not sure what you mean. JSON format makes processing rather easy, but it can do just normal text format, too, depending on what you need.
Can you be more specific? Perhaps provide your sample logs, what you are looking to do, and what you’ve tried.
Thank you for your reply…
Here’s my config.alloy file–>
local.file_match “log_files” {
path_targets = [{“path” = “/var/log/app-logs//.log”}]
sync_period = “5s”
}
local.file_match “log_gz_files” {
path_targets = [{“path” = “/var/log/app-logs//.log.gz”}]
sync_period = “5s”
}
loki.source.file “file_source” {
targets = local.file_match.log_files.targets
tail_from_end = false
forward_to = [loki.process.log_processing.receiver]
}
loki.source.file “file_gz_source” {
targets = local.file_match.log_gz_files.targets
tail_from_end = false
decompression {
enabled = true
initial_delay = “10s”
format = “gz”
}
forward_to = [loki.write.loki_receiver.receiver]
}
loki.process “log_processing” {
stage.json {
expressions = { output = “message”, response = “statusCode”, level = “level” }
}
stage.timestamp {
source = “timestamp”
format = “RFC3339”
}
forward_to = [loki.write.loki_receiver.receiver]
}
loki.write “loki_receiver” {
endpoint {
url = “http://loki:3100/loki/api/v1/push”
}
}
Actually, my json format of logs worked well.
The problem that I am having is by default, I am only having ‘filename’ label within my loki after pushing it.
How do I add other types of labels such as namespace, service, etc…, if any?
In the path that I have provided-> /var/log/app-logs//.{log,log.gz} => The first * represents the application name, and second * represents log files represented in the date format.
I believe the application name goes to the label ‘service’.
I am not sure what all other labels are there, and what exact values to give those labels.
- I want to know how I can establish a label ‘service’ within loki?
- I want to know what all labels are there and what values should be given to them?
Lastly, I am running grafana, loki, alloy as part of docker compose, there was only installation through docker within the alloy documentation.
Just correct the service of alloy that I am using, if it’s wrongly described.
alloy:
image: grafana/alloy:latest
container_name: alloy
ports:
- “12345:12345”
volumes:
- ./config.alloy:/etc/alloy/config.alloy
- /home/abhishek/Documents/logs:/var/log/app-logs:ro
command:
- run
- --server.http.listen-addr=0.0.0.0:12345
- --storage.path=/var/lib/alloy/data
- /etc/alloy/config.alloy
depends_on:
- loki
networks:
- grafana-network
Thank you in advance…
You can add any label you want, and there are several ways of doing it. You can add static label, or you can parse logs for something to set label as. I’d recommend you to read through loki.process documentation here for some examples: loki.process | Grafana Alloy documentation