Issue with regex pattern in scrape config for Grafana Loki and Promtail configuration deployed on K8S

Grafana Loki
,
1

Hi Team,

I am trying to add regex pattern to filter log and add labels by using Regex pattern in Scrape config of loki and promtail config as mentioned below

promtail:
enabled: true
config:
serverPort: 3101
clients:

  • url: http://{{ .Release.Name }}:3100/loki/api/v1/push
    log_level: debug
    snippets:
    scrapeConfigs: |
    - job_name: myapp-logs
    static_configs:
    - targets:
    - {{ .Release.Name }}
    labels:
    job: myapp-logs
    path: /usr/local/tomcat/logsMT/myapp.log
    pipeline_stages:
    - regex:
    expression: “(?P\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}) (?P\w+) (?P[^:]+):(?P[^\s]+) (?P\w+) (?P[^\s]+) Parameter:{(?P[^\}]+)} Status:(?P\d+) ExecutionTime:(?P[^\s]+)”
    - labels:
    level: level
    appname: appname
    appmodule: appmodule
    method: method
    endpoint: endpoint
    status: status
    - output:
    source: stdout

After deploying above configuration I am getting below error

"level=warn ts=2024-06-24T07:31:20.23667182Z caller=logging.go:123 msg="GET /ready (500) 156.257µs Response: “Not ready: Unable to find any logs to tail. Please verify permissions, volumes, scrape_config, etc.\n” ws: false; Accept: /; Connection: close; User-Agent: kube-probe/1.26;

However I am getting all the logs if I do not apply any regex pattern.

2

If you are collecting logs from a path, your target should be localhost, no?

3

yes I tried with localhost as well but still error is same.

4

Do you see any error log?

Also, try enable debugging and inspection mode and test your config on some sample logs and see what it says (see Troubleshooting Promtail | Grafana Loki documentation)

1 Like
5

I am trying now without regex but still I am not getting any log from application

Below are my configurations

server:
http_listen_port: 9080
grpc_listen_port: 0
positions:
filename: /var/log/positions.yaml
clients:
- url: http://grafana-loki-umatch:3100/loki/api/v1/push
scrape_configs:
- job_name: system
static_configs:
- targets:
- localhost
labels:
job: varlogs
path: /var/log/*log
- targets:
- localhost
labels:
job: sonarqube_logs
path: /opt/sonarqube/logs/*log

6
  1. Did you try the debug option?
  2. What are you seeing in your promtail logs?