LOKI - how set date of log

Grafana Loki
1

I would like to change date of log, which is sent to LOKI by promtail. Example of log lines, promtail and docker-compose is below.

Result

  • experimentally I added timestamp as label, and it was in LOKI
  • the captured group added added in section timestamp doesn’t work

Can you please help in this config? How to correct promtail file to have correct date in log?

2024-01-06 22:05:00 104.159.65.72 “GET /services HTTP/1.1” 106 12642
2024-01-06 22:06:00 104.159.65.73 “GET /services HTTP/1.1” 206 13642
2024-01-06 22:07:00 104.159.65.74 “GET /services HTTP/1.1” 306 14642
2024-01-06 22:07:00 104.159.65.75 “GET /services HTTP/1.1” 406 15642

promtail config like below:

server:
http_listen_port: 9080
grpc_listen_port: 0
log_level: “info”

positions:
filename: /tmp/positions.yaml

clients:
- url: http://loki:3100/loki/api/v1/push
tenant_id: docker

scrape_configs:

  • job_name: moc-logs
    static_configs:
    • targets:
      • localhost
        labels:
        env: dev
        job: moc-logs
        path: /var/log/moc-logs.txt
        pipeline_stages:
    • regex:
      expression: ‘^(?P<timestamp\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) (?P<ip\d+.\d+.\d+.\d+) “(?P<method[A-Z]+) (?P<path[^”]+)" (?P<status\d+) (?P<size\d+)’
    • timestamp:
      source: timestamp
      location: Europe/Warsaw
      format: ‘2024-01-06 22:05:00’
    • labels:
      timestamp:

------ docker compose

version: “3”

volumes:
grafana:
prometheus:
loki:

networks:
loki:

services:
loki:
image: grafana/loki:2.8.3
ports:
- “3100:3100”
volumes:
- ./config/loki2.yaml:/etc/loki/local-config.yaml
- ./test_data:/loki

command: -config.file=/etc/loki/local-config.yaml

-config.expand-env=true

networks:
  - loki

promtail:
image: grafana/promtail:2.8.3
volumes:
- ./test_logs/:/var/log/
- ./config/promtail.yaml:/etc/promtail/promtail.yaml
command: -config.file=/etc/promtail/promtail.yaml
networks:
- loki

2

When using a custom date format in promtail you need to use the reference time. See time package - time - Go Packages.

3

Thank you. It helped.

4

Can you please help how to write format when I have following date in line “21.103.19.181 - - [09/Jan/2024:23:37:23 +0000] “POST /infomediaries/implement HTTP/1.0” 401 8995”

This line is generted by container mingrammer/flog. How in general move date when we have time zone or would like to add offset?

5

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.