Promtail - inspect does not show the pipelines

Grafana Loki
1

I have this config:

scrape_configs:
- job_name: apache2
  pipeline_stages:
  - regex:
    expression: "^(?P<client>(?:[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}|(?:(?:[0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}|(?:(?:[0-9a-fA-F]{1,4}:)*[0-9a-fA-F]{1,4})?::(?:[0-9a-fA-F]{1,4}:)*[0-9a-fA-F]{1,4})))"
  static_configs:
  - targets: ['localhost']
    labels:
      job: apache2_logs
      __path__: /var/log/apache2/access.log

I run promtail --inspect, but I get no pipeline stages:

$ cat /var/log/apache2/access.log | promtail --stdin --config.file /etc/promtail/config.yml --inspect --dry-run
Clients configured:
----------------------
url: http://localhost:3100/loki/api/v1/push
batchwait: 1s
batchsize: 1048576
follow_redirects: true
enable_http2: true
backoff_config:
  min_period: 500ms
  max_period: 5m0s
  max_retries: 10
timeout: 10s
tenant_id: ""
drop_rate_limited_batches: false

2025-07-26T15:29:20.767333046+0200      {__path__="/var/log/messages", job="varlogs"}   127.0.0.1 - - [26/Jul/2025:00:09:14 +0200] "GET /server-status/?auto HTTP/1.1" 200 147 833 483 829 "-" "Prometheus-Apache-Exporter/1.0.9"
2025-07-26T15:29:20.76733639+0200       {__path__="/var/log/messages", job="varlogs"}   192.168.84.36 - - [26/Jul/2025:00:09:18 +0200] "GET /selfoss/items/sync?since=2025-07-25T10%3A31%3A16.000Z&tags=true&itemsStatuses=true HTTP/1.1" 403 2797 3950 91463 91648 "-" "Mozilla/5.0 (X11; Linux x86_64; rv:140.0) Gecko/20100101 Firefox/140.0"
2025-07-26T15:29:20.767343918+0200      {__path__="/var/log/messages", job="varlogs"}   192.168.84.27 - admin [26/Jul/2025:00:09:20 +0200] "GET /nextcloud/ocs/v2.php/apps/serverinfo/api/v1/info?format=json&skipApps=true&skipUpdate=true HTTP/1.1" 200 609 5639 1389975 1390358 "-" "nextcloud-exporter/"
2025-07-26T15:29:20.767371086+0200      {__path__="/var/log/messages", job="varlogs"}   127.0.0.1 - - [26/Jul/2025:00:09:29 +0200] "GET /server-status/?auto HTTP/1.1" 200 147 868 706 1377 "-" "Prometheus-Apache-Exporter/1.0.9"
2025-07-26T15:29:20.767377158+0200      {__path__="/var/log/messages", job="varlogs"}   192.168.84.27 - admin [26/Jul/2025:00:09:35 +0200] "GET /nextcloud/ocs/v2.php/apps/serverinfo/api/v1/info?format=json&skipApps=true&skipUpdate=true HTTP/1.1" 200 609 5658 368253 368424 "-" "nextcloud-exporter/"
2025-07-26T15:29:20.767416601+0200      {__path__="/var/log/messages", job="varlogs"}   192.168.84.36 - mdione [26/Jul/2025:00:09:40 +0200] "GET /nextcloud/ocs/v2.php/apps/notifications/api/v2/notifications?format=json HTTP/1.1" 304 2618 4497 119683 119842 "-" "Mozilla/5.0 (Linux) mirall/3.16.4-1 (Debian built) (Nextcloud, debian-6.12.33+deb13-amd64 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
2025-07-26T15:29:20.767838724+0200      {__path__="/var/log/messages", job="varlogs"}   192.168.84.36 - mdione [26/Jul/2025:00:09:40 +0200] "GET /nextcloud/ocs/v2.php/apps/user_status/api/v1/user_status?format=json HTTP/1.1" 200 2563 4653 220200 220458 "-" "Mozilla/5.0 (Linux) mirall/3.16.4-1 (Debian built) (Nextcloud, debian-6.12.33+deb13-amd64 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
2025-07-26T15:29:20.767841386+0200      {__path__="/var/log/messages", job="varlogs"}   192.168.84.36 - mdione [26/Jul/2025:00:09:40 +0200] "PROPFIND /nextcloud/remote.php/dav/files/mdione/ HTTP/1.1" 207 2700 4751 288933 289098 "-" "Mozilla/5.0 (Linux) mirall/3.16.4-1 (Debian built) (Nextcloud, debian-6.12.33+deb13-amd64 ClientArchitecture: x86_64 OsArchitecture: x86_64)"
2025-07-26T15:29:20.767856132+0200      {__path__="/var/log/messages", job="varlogs"}level=info ts=2025-07-26T13:29:20.76675468Z caller=promtail.go:135 msg="Reloading configuration file" sha3sum=bd5a7100afc5a9029bb99de4f870a31e2cbda16bd06984f8d592c620255aff88
level=warn ts=2025-07-26T13:29:20.767142348Z caller=stdin_target_manager.go:83 msg="too many scrape configs, skipping 1 configs."
level=error ts=2025-07-26T13:29:20.76844823Z caller=main.go:169 msg="error creating promtail" error="error creating loki server: listen tcp :9080: bind: address already in use"

In particular, the pipeline doesn’t seem to be being applied. Any ideas?

If it helps, this setup has already ingested lines before the regexp was set up. Also, yes, that’s not a whole regexp to parse apache logs.

too many scrape configs, skipping 1 configs maybe this? what could this be? Yes, there is another scrpe config, but it has another job_name.

2

It was a YAML issue. This is the final configuration:

scrape_configs:
- job_name: apache2
  pipeline_stages:
    - regex:
        expression: "^(?P<client>(?:[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}.[0-9]{1,3}|(?:(?:[0-9a-fA-F]{1,4}:){7}[0-9a-fA-F]{1,4}|(?:(?:[0-9a-fA-F]{1,4}:)*[0-9a-fA-F]{1,4})?::(?:[0-9a-fA-F]{1,4}:)*[0-9a-fA-F]{1,4}))) (?P<logname>(?:[a-zA-Z0-9_]+|-)) (?P<user>(?:[a-zA-Z0-9_]+|-)) \\[[^]]+\\] \"(?:(?P<method>.*) (?P<path>[^?]*)(?:\\?[^ ]*)? (?P<proto>.*)/(?P<proto_version>(?:[1-9][0-9]*|0).(?:[1-9][0-9]*|0))|(?P<trash>.*))\" (?P<status>(?:[1-9][0-9]*|0)) (?P<received>(?:[1-9][0-9]*|0)) (?P<sent>(?:[1-9][0-9]*|0)) (?P<ttfb>(?:(?:[1-9][0-9]*|0)|-)) (?P<request_time_us>(?:[1-9][0-9]*|0)) (?P<referer>\"(.*)\") (?P<user_agent>\"(.*)\")$"

  static_configs:
    - targets: ['localhost']
      labels:
        job: apache2_logs
        __path__: /var/log/apache2/access.log