Adding extra scraper jobs to promtail deployed with loki helm chart

Grafana Loki
,
1

hi everyone,

i am really having hard time to add extra jobs in promtail deployed with help of helm chart grafana/loki-stack.

i have tried to copy the values file from the helm chart

cp -p loki-stack/charts/promtail/values.yaml $OLDPWD/values_promtail_nginx_or.yaml

then added following conf to it:

- job_name: nginx
  static_configs:
  - targets:
      - localhost
    labels:
      app: nginx
      env: production
      host: ruan-prod-nginx
      __path__: /var/log/nginx/*.log
  pipeline_stages:
  - match:
      selector: '{app="nginx"}'
      stages:
      - regex:
          # logline example: 127.0.0.1 - - [21/Apr/2020:13:59:45 +0000] "GET /?foo=bar HTTP/1.1" 200 612 "http://example.com/lekkebot.html" "curl/7.58.0"
          expression: '^(?P<host>[\w\.]+) - (?P<user>[^ ]*) \[(?P<timestamp>.*)\] "(?P<method>[^ ]*) (?P<request_url>[^ ]*) (?P<request_http_protocol>[^ ]*)" (?P<status>[\d]+) (?P<bytes_out>[\d]+) "(?P<http_referer>[^"]*)" "(?P<user_agent>[^"]*)"?'
      - labels:
          host:
          method:
          request_url:
          status:
          user_agent:

then tried to upgrade the helm chart:

helm upgrade loki grafana/loki-stack --set resources.requests.cpu=100m --set resources.requests.memory=256Mi -f values_promtail_nginx_or.yaml

then deleted my promtail pods

kubectl delete pods -l app.kubernetes.io/name=promtail so they get re-deployed using the new config from values_promtail_nginx_or.yaml file.

however when i go to my promtail pod, the promtail.yaml conf is still the original one which came with initial deployment of loki helm chart.

I’m seeking guidance on how to update the promtail.yaml configuration file to parse nginx logs. My goal is to break down these logs into multiple fields, which will enable me to construct Grafana dashboards based on the values of these fields. Any assistance with this process would be greatly appreciated.

THanks in advance

2

This topic was automatically closed 365 days after the last reply. New replies are no longer allowed.