Loki Timestamp, LogQL Query issues

Hi,

I’m new to my PLG, and facing some issues with the monitoring, please suggest on this.

my log format is like below - which has utc time as timestamp

timedurationipusernamemethodmessage

ex: 2021-09-27T06:39:42.1756Z 200 00:00.001 1.1.17.110 testuser GET this is log message

and below is the promtail configuration

server:
http_listen_port: 9080
grpc_listen_port: 0

positions:
filename: E:/Logging/tmp/promtail/positions.yaml

clients:

  • url: http://localhost:3100/loki/api/v1/push
    scrape_configs:
  • job_name: API Activity
    static_configs:
    • targets:
      • localhost
        labels:
        job: APIActivity
        path: “E:/Logging/API/**/activity.log”
        host: “server1”
        pipeline_stages:
    • regex:
      expression: “^(?s)(?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P.*)$”
    • labels:
      duration:
      responseCode:
      ipAddr:
      user:
    • timestamp:
      source: time
      format: “RFC3339”

Issues:

  1. The timestamp showing in grafana dashboard is time in which the logs have added to Loki.
  2. old logs are not processing, only new ones are getting displayed in grafana.
  3. how to calculate (LogQL query) the average duration of all calls in each server (host label) ?

thanks in advance,
JK

Hi,

on first sight I’d say that expression: “^(?s)(?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P\S+?) (?P.*)$” is missing the named group time.