How to get alloy to parse /var/log/syslog correctly?

Grafana Alloy
17

That’s a long discussion, so I am just going to look at your first message.

Some basics first:

  • Unless you instruct Alloy to parse content, it just takes the log line, put a timestamp when grabbed and push it to Loki with “zero” knowledge about the content
  • Loki doesn’t parse stuff either on ingestion. You can store log lines with zero knowledge about what’s inside
  • Content is parsed at query time (using pattern or logfmt or json, etc)
  • The typical recommendation is to add labels at Alloy level about what’s AROUND and not INSIDE the logs (ex: hostname=my-server, logfile=/var/log/apache.log, env=prod, job=system)

After that said, your usecase is not “weird”: your logline embed a timestamp that can be different from the time the logline is written or grabbed.

First of all: does it matter to correct your timestamp ? Order is preserved, data is correct, only problem you have is that Loki thinks it happened 0.3s later than normally. If it doesn’t matter, just give up on that. You would add a lot of config for no real value.

If you still want to proceed, you have 2 different formats in your logs (one with ts, one without) and you expected 2 differents results. You need to create 2 pipelines in Alloy (you can check localhost:12345 to see your current graph). You can find some knowledge about timestamp replacement here (it uses promtail but that the same concept using Alloy semantic)

3 Likes