Promtail does not drop logs on relabel_configs action: drop

Grafana Loki
1

I have the following promtail configuration, that should do nothing:

scrape_configs:
  - job_name: "docker"
    docker_sd_configs:
      - host: "unix:///var/run/docker.sock"
        refresh_interval: "2s"
    pipeline_stages:
      - docker: {}
      - static_labels:
          source: test
    relabel_configs:
       - action: drop

Yet, it transfers the logs of all docker containers. Why? Shouldn’t action:drop drop the target from promtail? How to postprocess targets to drop them depending on

2024-04-09T08:40:04-04:00 {dc="wee-dev", instance="weelxavt011d", source="test"} level=error ts=2024-04-09T12:40:04.028387352
Z caller=target.go:100 target=docker/9e961738d7de8981c4160f8791e18833c57906d5fc87f07bf140eae0adbd0983 msg="could not inspect
container info" container=9e961738d7de8981c4160f8791e18833c57906d5fc87f07bf140eae0adbd0983 err="Error response from daemon: N
o such container: 9e961738d7de8981c4160f8791e18833c57906d5fc87f07bf140eae0adbd0983"
2024-04-09T08:40:04-04:00 {dc="wee-dev", instance="weelxavt011d", source="test"} Tue Apr  9 12:40:04 UTC 2024
2024-04-09T08:40:09-04:00 {alloc_id="0e0ade1b-9ed1-2928-3c57-b2205cc5d6ca", dc="wee-dev", group="fabio", instance="weelxavt01
1d", job="fabio", namespace="services", source="test", task="fabio"} level=info ts=2024-04-09T12:40:09.027543776Z caller=targ
et_group.go:128 msg="added Docker target" containerID=33782221390e44ac72bcec36d3afab82cb150de040c2d4aabe0ff06c6dda886a
2024-04-09T08:40:09-04:00 {dc="wee-dev", instance="weelxavt011d", source="test"} level=info ts=2024-04-09T12:40:09.187221626Z
 caller=target.go:132 target=docker/33782221390e44ac72bcec36d3afab82cb150de040c2d4aabe0ff06c6dda886a msg="finished transferri
ng logs" written=61 container=33782221390e44ac72bcec36d3afab82cb150de040c2d4aabe0ff06c6dda886a
2

relabel_configs is for overwriting labels, I believe. if you are looking to drop logs, try the drop action (see drop | Grafana Loki documentation)

3

Hi, thank you.

The problem with drop action is that it analyzes every single line in the file.

I have the following config:

static_configs:
  - labels:
      __path__: '/some/path/[0-9]*'

Some of the files are bz2 some are not. I would want to exclude .bz2 files from analysis. I know that I can do:

pipeline_stages:
  - regex:
      source: filename
      expression: '^.*(?P<match>.bz2)$'
  - labels:
      match:
  - match:
     selector: '{match=~".+"}'
     action: drop
  - labeldrop: [match]

However, pipeline_stages analysis every single line of the file, which doesn’t make much sense for bz2 files.

Ideally I would wan tto:

relabel_configs:
   - source_labels: [filename] 
     regex: '^.*.bz2$'
     action: drop

However that does not work.

Thank you.

4
  1. Usually you don’t need to drop logs based on filename, because you should have configured promtail to not read them in the first place. For emaple, let’s say you have:
/var/log/test/this-is-a-log.log
/var/log/test/this-is-not-a-log.bz

You should configure your path with /var/log/test/*.log to avoid reading .bz file in the first place.

  1. If you can’t help it and those have to be read, you can still use the drop action. In your previous comment you are attempting to drop the label, not the log. If you are looking to drop logs, it would looks something like this (not tested):
- match:
      selector: '{filename=~"^.+\.bz2$"}'
      action: drop
      drop_counter_reason: bz_log
1 Like