Loki Timestamp, LogQL Query issues

[jkevepro]

Hi,

I’m new to my PLG, and facing some issues with the monitoring, please suggest on this.

my log format is like below - which has utc time as timestamp

timedurationipusernamemethodmessage

ex: 2021-09-27T06:39:42.1756Z 200 00:00.001 1.1.17.110 testuser GET this is log message

and below is the promtail configuration

server:
http_listen_port: 9080
grpc_listen_port: 0

positions:
filename: E:/Logging/tmp/promtail/positions.yaml

clients:

• url: http://localhost:3100/loki/api/v1/push
  scrape_configs:
• job_name: API Activity
  static_configs:
  • targets:
    • localhost
      labels:
      job: APIActivity
      path: “E:/Logging/API/**/activity.log”
      host: “server1”
      pipeline_stages:
  • regex:
    expression: “^(?s)(?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P.*)$”
  • labels:
    duration:
    responseCode:
    ipAddr:
    user:
  • timestamp:
    source: time
    format: “RFC3339”

Issues:

1. The timestamp showing in grafana dashboard is time in which the logs have added to Loki.
2. old logs are not processing, only new ones are getting displayed in grafana.
3. how to calculate (LogQL query) the average duration of all calls in each server (host label) ?

thanks in advance,
JK

[karsten]

Hi,

on first sight I’d say that expression: “^(?s)(?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P.*)$” is missing the named group time.