Match timestamp at the beginning of a log line

Grafana Alloy
1

I’m sure this question has been asked already in various contexts, but I’m still not very sure how to do this as practically and efficiently as possible: how can I match a timestamp at the beginning of log line like the following:

Wed Jul  9 19:43:33 2025 fullname/192.168.50.43:49275 Incoming Data Channel: Cipher 'AES-128-GCM' initialized with 128 bit key

Which is, I guess, the syslog timestamp (+ the year, which isn’t part of the syslog timestamp, as far as I know)

I see that alloy allows for stage.regex from which I could extract the timestamp using stage.timestamp. But isn’t that overkill? Loki’s line pattern, for example, would alloy me to simply match the first space-separated strings, so in this case that would just be the first 5 words. Doesn’t alloy allow for something as simple as that which wouldn’t take up so much processing power as regex normally require?

2

The short answer is you don’t.

Each log line, when ingested into Loki, has a timestamp associated with the log line, in the epoch format to the nanosecond precision. this can be different from whatever timestamp you have in your log line, depending on when the logs are accepted by Loki.

It is far easier to rely on the timestamp created by Loki for each log line and let Loki handle the time matching, instead of trying to do that yourself. Therefore, it is your responsibility to ensure the timestamp is correct when logs are sent to Loki by parsing and setting it if needed.

3

So you mean to say that this timestamp associated with the log line is not added when ingested by Loki, instead it’s added when (in this case) Alloy (so by Alloy) sends the logs to Loki?

No, I misunderstood. You’re saying I should rely on the timestamp that is created when Loki receives the logs, so by Loki itself.

So if something happens in the meantime, for some reason logs cannot be sent for an hour or so, the timestamp will be completely skewed… I understand you’re supposed to now let that happen, but that doesn’t sound like a robust solution at all. The point is to be as consistent and as safe as possible.

Therefore, it is your responsibility to ensure the timestamp is correct when logs are sent to Loki by parsing and setting it if needed.

I have no idea how this conclusion follows from the previous sentence where you say that you’re supposed to let Loki do it.
So, ok, if it is my responsibility to parse the logs, I will ask the question again, how can I do this as efficiently as possible on the alloy-side? With regex? There’s no other way?

4

Sorry if my previous comment may have been confusing. Let me rephrase:

When you send a log to Loki, whether through API or log agent such Alloy, you have to have a timestamp associated with the API call (see Loki HTTP API | Grafana Loki documentation).

Now, with log agents such Alloy, you can either:

  1. Parse the log line (in your example it would be Wed Jul 9 19:43:33 2025) and turn that into epoch timestamp.
  2. Don’t parse.

If you don’t parse, then Alloy will use the time when it processes the log, which can be off anywhere from seconds to minutes) when it sends the log line to Loki.

I would usually recommend parsing for the timestamp.

5

I understand now.

So if alloy itself sends the timestamp, then I would guess there’s a much lower chance that the timestamp will be off than if everything relied exclusively on loki (on the time when loki received the logs). So that’s not such a bad compromise after all, but I see your point.

I’ve misread again, but I’m sure it was my fault, at least this time :smiley: You said it is associated with the API call, so that means there’s no timestamp added before by Alloy, it’s added during this call. If Loki isn’t available at that point, then I might get into trouble and the timestamp might skew.

So, for instance, if I were to use something like this in alloy:

loki.process "ovpn" {
        stage.regex {
                expression = `^(?P<log_timestamp>\w{3}\s\w{3}\s{2}\d{1,2}\s\d{2}:\d{2}:\d{2}\s\d{4})`
        }

        stage.timestamp {
                source = "log_timestamp"
                format = "Mon Jan _2 15:04:05 2006"
        }

        forward_to = [loki.write.endpoint.receiver]
}

Would the stage.timestamp itself override the timestamp that Alloy attaches automatically?

And if that’s the case, how would I know if this is really happening and if it isn’t simply just being ignored?

That said, let’s say I want to match the next strings (the fullname/IP, whatever), how can I do that without having to repeat the regex in stage.regex? So basically to be as practical as possible.

Later edit:
Related to the last point, now that I think about, I guess I could just continue the regex and create labels through the regex named groups and that’s about it, I suppose…

6

Yes, that’s what you want to do if you want to make sure the timestamp matches your logs.

1 Like