Drop all logs except two name spaces in promtail

Grafana Loki
1

I am using the below promtail configuration

I need to drop all logs except 2 namespaces. But still full logs are coming ?
promtail:
config:
lokiAddress: loki-distributed-gateway
snippets:
common:
- action: replace
source_labels:
- __meta_kubernetes_pod_node_name
target_label: node_name

        - action: replace
          source_labels:
            - __meta_kubernetes_namespace
          target_label: namespace
        - action: replace
          replacement: $1
          separator: /
          source_labels:
            - namespace
            - app
          target_label: job
        - action: replace
          source_labels:
            - __meta_kubernetes_pod_name
          target_label: pod

        - action: replace
          regex: true/(.*)
          replacement: /var/log/pods/*$1/*.log
          separator: /
          source_labels:
            - __meta_kubernetes_pod_annotationpresent_kubernetes_io_config_hash
            - __meta_kubernetes_pod_annotation_kubernetes_io_config_hash
            - __meta_kubernetes_pod_container_name
          target_label: __path__
      pipelineStages:
        - docker: { }
        - multiline:
            firstline: '^[[:space:]]*\[java\] \d{4}-\d{1,}-\d{1,} \d{1,}:\d{1,}:\d{1,},\d{1,}
            [[:graph:]]* [A-Z]{4,}[[:space:]]*[[:graph:]]* - '
      scrapeConfigs: |-
        # Pods with a label 'app.kubernetes.io/name'
        - job_name: kubernetes-pods-app-kubernetes-io-name
          pipeline_stages:
            {{- toYaml .Values.config.snippets.pipelineStages | nindent 4 }}
          kubernetes_sd_configs:
            - role: pod
          relabel_configs:
            - action: replace
              source_labels:
                - __meta_kubernetes_pod_label_app_kubernetes_io_name
              target_label: app
            - action: drop
              regex: ''
              source_labels:
                - app
            - action: replace
              source_labels:
                - __meta_kubernetes_pod_label_app_kubernetes_io_component
              target_label: component
            {{- if .Values.config.snippets.addScrapeJobLabel }}
            - action: replace
              replacement: kubernetes-pods-app-kubernetes-io-name
              target_label: scrape_job
            {{- end }}
            {{- toYaml .Values.config.snippets.common | nindent 4 }}

what am I doing wrong here

also when we parse the logs, its not detecting new keyvalu pairs other than the labels we have specified in the config

2

Is that your entire promtail configuration? You seem to have two pipeline stages, and the only drop action I see is evaluated against app label.

This is how I usually drop logs:

scrape_configs:
- job_name: something
  static_configs:
  <CONFIG...>

  pipeline_stages:
  <OTHER_CONFIG...>
  - match: # Drop logs that don't match
      selector: '{label!="value"}'
      action: drop
      drop_counter_reason: non_essential_log
  - match:
      selector: '{label="value"}'
      stages:
      <OTHER_STAGES>
1 Like