Promtail pipeline not sending labels to Loki

Grafana
1

Hello,

I’m trying to get Promtail to pipe some logs into Loki and I’m having trouble getting the pipeline stages to add the labels I want. I’m reading W3C IIS logs on Windows Server 2016 but it doesn’t seem to add the labels that I want to dynamically add in Loki.

My config is:

server:
http_listen_port: 9080
grpc_listen_port: 0

positions:
filename: D:\APPS\promtail\positions.yaml

clients:

scrape_configs:

  • job_name: W3SVC1
    entry_parser: raw
    static_configs:
    • targets:
      • localhost
        labels:
        job: W3SVC1
        host: SD24RTVAP6023
        env: pc_server
        path: C:\inetpub\logs\LogFiles\W3SVC1*.log
        pipeline_stages:
    • match:
      selector: ‘{job=“W3SVC1”}’
      stages:
      • regex:
        expression: “^(?P[\d\-]{10} [\d:]{8}) (?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) (?P\w+) (?P[^\s]+) (?P[^\s]+) (?P\d+) (?P[^\s]+) (?P<requestor_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) (?P[^\s]+) (?P[^\s]+) (?P<status_code>\d+) (?P\d+) (?P<win32_status>\d+) (?P<time_taken>\d+)(?P)$”
      • timestamp:
        source: timestamp
        format: 2019-07-30 15:27:11
      • labels:
        method:
        status_code:

When I run a dry run on one of the files I get the labels that I want, for the most part.
Example dry run:

PS D:\APPS\promtail> cat C:\inetpub\logs\LogFiles\W3SVC1\u_extend1.log | .\promtail-windows-amd64.exe --stdin --config.file=.\promtail.yaml --dry-run
Clients configured:

url: http://SD24RTVAP6024.TST.DCE-EIR.NET:3100/loki/api/v1/push
batchwait: 1s
batchsize: 102400
backoff_config:
min_period: 500ms
max_period: 5m0s
max_retries: 10
timeout: 10s
tenant_id: “”

level=info ts=2020-10-28T16:29:33.1158711Z caller=server.go:194 http=[::]:9080 grpc=[::]:60186 msg=“server listening on addresses”
level=info ts=2020-10-28T16:29:33.1168672Z caller=main.go:108 msg=“Starting Promtail” version=“(version=1.6.0, branch=HEAD, revision=6e3e2fa3)”
2020-10-28T12:29:33 {path=“C:\inetpub\logs\LogFiles\W3SVC1\.log", env=“pc_server”, host=“SD24RTVAP6023”, job=“W3SVC1”, method=“POST”, status_code=“200”} 2020-10-28 14:30:16 7.32.33.33 POST /PCS/LabService.svc - 443 - 7.32.33.29 Apache+CXF+2.5.9 - 200 0 0 458
2020-10-28T12:29:33 {path="C:\inetpub\logs\LogFiles\W3SVC1\.log”, env=“pc_server”, host=“SD24RTVAP6023”, job=“W3SVC1”, method=“POST”, status_code=“200”} 2020-10-28 14:31:07 7.32.33.33 POST /PCS/TaskManagementService.svc - 443 - 7.32.33.29 Apache+CXF+2.5.9 - 200 00 546
2020-10-28T12:29:33 {path=“C:\inetpub\logs\LogFiles\W3SVC1\.log", env=“pc_server”, host=“SD24RTVAP6023”, job=“W3SVC1”, method=“POST”, status_code=“200”} 2020-10-28 14:31:16 7.32.33.33 POST /PCS/LabService.svc - 443 - 7.32.33.29 Apache+CXF+2.5.9 - 200 0 0 15
2020-10-28T12:29:33 {path="C:\inetpub\logs\LogFiles\W3SVC1\.log”, env=“pc_server”, host=“SD24RTVAP6023”, job=“W3SVC1”, method=“POST”, status_code=“200”} 2020-10-28 14:32:08 7.32.33.33 POST /PCS/TaskManagementService.svc - 443 - 7.32.33.29 Apache+CXF+2.5.9 - 200 0 42
2020-10-28T12:29:33 {path=“C:\inetpub\logs\LogFiles\W3SVC1\.log", env=“pc_server”, host=“SD24RTVAP6023”, job=“W3SVC1”, method=“POST”, status_code=“200”} 2020-10-28 14:32:17 7.32.33.33 POST /PCS/LabService.svc - 443 - 7.32.33.29 Apache+CXF+2.5.9 - 200 0 0 31
2020-10-28T12:29:33 {path="C:\inetpub\logs\LogFiles\W3SVC1\.log”, env=“pc_server”, host=“SD24RTVAP6023”, job=“W3SVC1”, method=“POST”, status_code=“200”} 2020-10-28 14:33:17 7.32.33.33 POST /PCS/LabService.svc - 443 - 7.32.33.29 Apache+CXF+2.5.9 - 200 0 0 15
2020-10-28T12:29:33 {path=“C:\inetpub\logs\LogFiles\W3SVC1\.log", env=“pc_server”, host=“SD24RTVAP6023”, job=“W3SVC1”, method=“POST”, status_code=“200”} 2020-10-28 14:34:09 7.32.33.33 POST /PCS/TaskManagementService.svc - 443 - 7.32.33.29 Apache+CXF+2.5.9 - 200 0 35
2020-10-28T12:29:33 {path="C:\inetpub\logs\LogFiles\W3SVC1\.log”, env=“pc_server”, host=“SD24RTVAP6023”, job=“W3SVC1”, method=“POST”, status_code=“200”} 2020-10-28 14:34:17 7.32.33.33 POST /PCS/LabService.svc - 443 - 7.32.33.29 Apache+CXF+2.5.9 - 200 0 0 15

I’ve tried a few different configs but they look like most of the examples I’ve seen and haven’t read anything about this. What am I missing?

2

can you share your query?

3

Is your problem solved? if so can you send your config code

1 Like
4

I did sorta fix the issue. Most of it was me not reading the manual well enough and using things I thought should work when the documentation told me that it shouldn’t be done that way. My current issue now is Loki sometimes stops reporting, but promtail seems to work fine now.

Updated Config:

server:
http_listen_port: 9080
grpc_listen_port: 0

positions:
filename: D:\APPS\promtail\positions.yaml

clients:

scrape_configs:

  • job_name: W3SVC1
    entry_parser: raw
    static_configs:
    • targets:
      • localhost
        labels:
        job: W3SVC1
        host: SD24RTVAP6023
        env: pc_server
        path: C:\inetpub\logs\LogFiles\W3SVC1*.log
        pipeline_stages:
    • match:
      selector: ‘{job=“W3SVC1”}’
      stages:
      • regex:
        expression: “^(?P[\d\-]{10} [\d:]{8}) (?P\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) (?P\w+) (?P[^\s]+) (?P[^\s]+) (?P\d+) (?P[^\s]+) (?P<requestor_ip>\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}) (?P[^\s]+) (?P[^\s]+) (?P<status_code>\d+) (?P\d+) (?P<win32_status>\d+) (?P<time_taken>\d+)(?P)”
      • timestamp:
        source: timestamp
        format: 2006-01-02 15:04:05
      • labels:
        method:
        status_code:
  • job_name: w3wp
    entry_parser: raw
    static_configs:
    • targets:
      • localhost
        labels:
        job: w3wp
        host: SD24RTVAP6023
        env: pc_server
        path: D:\APPS\PCSERVER\orchidtmp\LTLogger\w3wp*.txt
        pipeline_stages:
    • match:
      selector: ‘{job=“w3wp”}’
      stages:
      • regex:
        expression: “^(?P[\d-]{10} [\d:,]{12})\s+\[(?P<error_code>\d+)\]\s+\[(?P<sub_error_code>\d+)\]\s+(?P\w+)\s+(?P.*)”
      • timestamp:
        source: timestamp
        format: 2006-01-02 15:04:05,000
      • labels:
        error_code:
        level:
6

(post withdrawn by author, will be automatically deleted in 24 hours unless flagged)

7

HI, I am trying this but by log has UTC time , how you managed that.