Logs from Promtail on Windows are too verbose in Grafana

Grafana Loki
4

You’re right — breaking up or modifying JSON strings in Promtail is not easy and should be avoided when possible. To preserve the original log structure, I’ve adopted the line_format approach in LogQL queries to display only the relevant information.

Sample Query

{logsource="windows-eventlog", job="windows-security", username!="", ip!=""}
| json
| event_id == 4625
| line_format "- Login failed: user {{ .username }} from IP {{ .ip }} to {{ .computer }} ({{.destination_ip}})"

Sample promtail-config.yaml

server:
  http_listen_port: 9090
  grpc_listen_port: 0

positions:
  filename: C:\promtail\positions.yaml

clients:
  - url: http://<ip-loki>:<port>/loki/api/v1/push

scrape_configs:
  - job_name: windows-security
    windows_events:
      locale: 1033
      use_incoming_timestamp: true
      exclude_event_data: false
      exclude_user_data: false
      bookmark_path: C:\promtail\bookmark_security.xml
      eventlog_name: "Security"
      xpath_query: '*'
      labels:
        logsource: windows-eventlog
        job: windows-security
        type: security
        hostname: ${COMPUTERNAME}
        destination_ip: Your-server-IP

    pipeline_stages:
      - json:
          expressions:
            event_data: event_data

      - template:
          source: event_data
          template: '{{ .Value | replace "\\u003c" "<" | replace "\\u003e" ">" }}'
          output: event_data

      - regex:
          source: event_data
          expression: "<Data Name='TargetUserName'>(?P<username>[^<]*)</Data>.*?<Data Name='IpAddress'>(?P<ip>[^<]*)</Data>.*?<Data Name='TargetDomainName'>(?P<computer>[^<]*)</Data>"

      - labels:
          username:
          ip:
          computer:

This way, the log remains intact, and parsing is simplified. For alerting or dashboard use cases, I extract only necessary fields into labels using pipeline_stages, and only when absolutely required.

Thank you for the suggestion @tonyswumac