Been trying to parse the timestamp from logs using pipeline_stages with the following configs
pipeline_stages:
- match:
selector: '{job="varlogs"}'
stages:
- regex:
expression: '.(?P<ts>\d{2}-\d{2}-\d{4}\s\d{2}:\d{2}:\d{2}). .(?P<level>\w*). *(?P<msg>hello)'
- labels:
level:
- timestamp:
source: ts
format: "02-01-2006 15:04:05"
- output:
source: msg
The following single line log can be parsed properly,
[30-06-2021 04:18:21] [info] hello
But if the hour is more than 12, then the parsing is failed. For example, parsing the following log shows error “hour out of range”
[30-06-2021 14:18:21] [info] hello
cat test.log | promtail --stdin -log.level=debug --dry-run --config.file=test.yaml
Clients configured:
----------------------
url: http://loki:3100/loki/api/v1/push
batchwait: 1s
batchsize: 1048576
follow_redirects: false
backoff_config:
min_period: 500ms
max_period: 5m0s
max_retries: 10
timeout: 10s
tenant_id: ""
level=debug ts=2021-07-28T05:08:12.021519856Z caller=manager.go:58 msg="configured to read from stdin"
level=debug ts=2021-07-28T05:08:12.021784902Z caller=regex.go:132 component=pipeline component=stage type=regex msg="extracted data debug in regex stage" extracteddata="map[__path__:/log/*log job:varlogs level:info msg:hello timestamp:30-06-2021 14:18:21]"
level=debug ts=2021-07-28T05:08:12.021821869Z caller=timestamp.go:196 component=pipeline msg="failed to parse time" err="parsing time \"30-06-2021 14:18:21\": hour out of range" format="02-01-2006 03:04:05" value="30-06-2021 14:18:21"
2021-07-28T05:08:12.021716154+0000 {__path__="/log/*log", job="varlogs", level="info"} hello
I tested the logic in a simple go program and it seems working fine.
layout := "02-01-2006 15:04:05"
str := "30-06-2021 14:18:21"
t, err := time.Parse(layout, str)
if err != nil {
fmt.Println(err)
}
fmt.Println(t)
# output
# 2021-06-30 14:18:21 +0000 UTC
Not quite sure where the problem is?