# Loki/Promtail: Parse timestamp in pipeline doesn't work **URL:** \ **Category:** Grafana Loki\ **Tags:** loki\ **Created:** [January 28, 2020, 7:10pm UTC](https://community.grafana.com/t/loki-promtail-parse-timestamp-in-pipeline-doesnt-work/25236 "2020-01-28T19:10:12Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![bennigraf](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/bennigraf/32/9397_2.png) [@bennigraf](https://community.grafana.com/u/bennigraf)\ **Post date:** [January 28, 2020, 7:10pm UTC](https://community.grafana.com/t/loki-promtail-parse-timestamp-in-pipeline-doesnt-work/25236/1 "2020-01-28T19:10:12Z")
Hi everyone, I’m evaluating Loki for analyzing log files right now and have problems parsing the datetime that’s part of an apache access log line. The log line looks something like that (ip address changed): ``` 10.11.12.0 - - [28/Jan/2020:00:27:21 +0100] "GET / HTTP/1.1" 200 2963 "-" "worldping-api" ``` My regex looks like this: ``` - regex: expression: '(?P\w{1,3}.\w{1,3}.\w{1,3}.\w{1,3}) - - .(?P\d{2}.*\d{4}). "(?P.*)" (?P\d+) .* "(?P.*)" "(?P.*)"' ``` This seems to work; when setting up labels for all those matches they show up in Grafana explorer, and the `timestamp` field contains `28/Jan/2020:18:29:22 +0100`. But Promtail doesn’t seem to parse the datetime correctly, since in Grafana each log entry shows up with the time of when it was recorded (which doesn’t correlate to when the request was made, since the logs are written to the file with a delay). I setup a timestamp parser like this: ``` - timestamp: source: timestamp format: "02/Jan/2006:03:04:05 -0700" ``` I don’t have much Go experience, but tried to setup the format string according to the `time.Parse` docs and validate it with [http://gotime.agardner.me/parser.html](http://gotime.agardner.me/parser.html) – according to this, the format string should be allright. Can somebody give me a hint about why this could not work and how to improve it? Best, Benjamin. For reference, my whole pipeline stage looks like this: ```auto pipeline_stages: - match: selector: '{filename="/home/alu/logs/access_log"}' stages: - regex: expression: '(?P\w{1,3}.\w{1,3}.\w{1,3}.\w{1,3}) - - .(?P\d{2}.*\d{4}). "(?P.*)" (?P\d+) .* "(?P.*)" "(?P.*)"' - labels: clientAddr: request: code: referrer: client: timestamp: - timestamp: source: timestamp format: "02/Jan/2006:03:04:05 -0700" ``` ---
**Author:** ![bennigraf](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/bennigraf/32/9397_2.png) [@bennigraf](https://community.grafana.com/u/bennigraf)\ **Post date:** [January 29, 2020, 8:06am UTC](https://community.grafana.com/t/loki-promtail-parse-timestamp-in-pipeline-doesnt-work/25236/2 "2020-01-29T08:06:36Z")
Hey all, sooo I figured it out in the mean time. It turns out that `03` in the time format string references 12h (am/pm) format, whereas my logfile contains the hour of the day in 24h format. Thus the format string to parse the apache access log datetime has to look like this: `02/Jan/2006:15:04:05 -0700` (notice the `15` instead of `03`). I was able to debug this by running promtail with `-log.level=debug` (and a very short demo log file), which I read about here: [https://github.com/grafana/loki/issues/1330#issuecomment-559713373](https://github.com/grafana/loki/issues/1330#issuecomment-559713373) Thanks to all contributors for this nice piece of software! Best, Benjamin. ---
**Author:** ![medmex](https://sea2.discourse-cdn.com/grafana/user_avatar/community.grafana.com/medmex/32/25166_2.png) [@medmex](https://community.grafana.com/u/medmex)\ **Post date:** [August 12, 2022, 9:43am UTC](https://community.grafana.com/t/loki-promtail-parse-timestamp-in-pipeline-doesnt-work/25236/3 "2022-08-12T09:43:44Z")
I have a log line like this ```auto 1.1.1.1 - - [23/Jul/2022:14:24:00 +0200] "GET /app/login?return=x HTTP/1.0" 200 2975 "-" "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/103.0.0.0 Safari/537.36" **0/177102** ``` And promtail setting like ```auto scrape_configs: - job_name: system pipeline_stages: - regex: expression: '^(?P\S+) (?P\S+) (?P\S+) \[(?P[\w:/]+\s[+\-]\d{4})\] \"?(?P\S+)\s?(?P\S+)?\s?(?P\S+)?\" (?P\d{3}|-) (?P\d+|-)\s?\"?(?P[^\"]*)\"?\s?\"?(?P[^\"]*)?\" ?(?P\S+)?' - labels: action: status_code: date: time: ip: timestamp: - timestamp: source: timestamp format: "02/Jan/2006:15:04:05 -0700" static_configs: - targets: - localhost labels: job: apache env: dev __path__ : /var/log/access_ssl_log.processed* ``` This should parse the timestamp correctly. But when i go into Grafana there is no data, even if i select the whole Year. If i remove the timestamp and let it set the timestamp the time of scraping then in grafana i can see the data but it’s timestmap is the time of the parsing and not the time of the page visit (this is apache log) ---
**Author:** ![jaisonvj053](https://avatars.discourse-cdn.com/v4/letter/j/f1d935/32.png) [@jaisonvj053](https://community.grafana.com/u/jaisonvj053)\ **Post date:** [February 27, 2024, 4:59pm UTC](https://community.grafana.com/t/loki-promtail-parse-timestamp-in-pipeline-doesnt-work/25236/4 "2024-02-27T16:59:21Z")
Me to facing same issue let me know if you get solution ---
**Author:** ![hakanvansbro](https://avatars.discourse-cdn.com/v4/letter/h/aca169/32.png) [@hakanvansbro](https://community.grafana.com/u/hakanvansbro)\ **Post date:** [July 24, 2024, 7:08am UTC](https://community.grafana.com/t/loki-promtail-parse-timestamp-in-pipeline-doesnt-work/25236/5 "2024-07-24T07:08:26Z")
Same issue for me; if successfully parsing a timestamp from my log with promtail I could not put a query to loki to show the logline. However, loki will hold the labels extraced from the log lines, but any of my time range attempts will not show the log lines… using grafana/loki:2.9.2 grafana/promtail:2.9.2 I can provide more details if needed. Meanwhile I dig a bit further. Just a short note on how I inspect promtails parsing: ```auto # dry-run command line head -n 2 test.txt | ../promtail-linux-amd64 --config.file ../promtail-local-config.yaml --stdin --dry-run --inspect ``` ```auto # without timestamp parsing, i.e. adding 'now' as timestamp 2024-07-24T09:01:30.222017626+0200 { __path__ ="/etc/promtail/pizza.txt", httpmethod="POST", job="pizzalogs", logtype="INF", prot="HTTP/1.1", returncode="200", ts="2024-07-22 00:00:12.746 +01:00"}2024-07-22 00:00:12.746 +01:00 [INF] Request finished ``` ```auto # with timestamp parsing, using the date in the log line [inspect: timestamp stage]: {stages.Entry}.Entry.Entry.Timestamp: -: 2024-07-24 09:02:33.378827674 +0200 CEST +: 2024-07-22 00:00:12.746 +0100 +0100 2024-07-22T00:00:12.746+0100 { __path__ ="/etc/promtail/pizza.txt", httpmethod="POST", job="pizzalogs", logtype="INF", prot="HTTP/1.1", returncode="200", ts="2024-07-22 00:00:12.746 +01:00"}2024-07-22 00:00:12.746 +01:00 [INF] Request finished ```