Limiting Promtail to a specific namespace

Grafana Loki
,
1

Hi,
I’m a bit new to Loki and i need some help. I want to ship only a specific k8s namespace (kube-system) to Loki using Pormtail.
I’m trying to limit the Promtail to this namespace using regex. However, i still see other namespaces in the logs. What could be the issue?

---
config:
  positions:
    filename: /run/promtail/positions.yaml
  clients:
    - url: http://loki-gateway/loki/api/v1/push
      tenant_id: 1
  snippets:
    extraScrapeConfigs: |
      - job_name: kubernetes-pods-kube-system
        kubernetes_sd_configs:
          - role: pod
        pipeline_stages:
          - docker: {}
          - cri: {}
          - json:
              expressions:
                message: message
        relabel_configs:
          # Select only pods from the kube-system namespace
          - action: keep
            source_labels: [__meta_kubernetes_namespace]
            regex: kube-system
          # Ensure that only container logs are scraped from the correct log path
          - source_labels: [__meta_kubernetes_pod_container_name]
            target_label: __path__
            replacement: /var/log/pods/*/*/*.log
2
  1. Make sure __meta_kubernetes_namespace is actually captured.
  2. Try moving the keep action to later part in the pipeline configuration.
4
  1. How can i know if __meta_kubernetes_namespace is actually captured? I tried enabling debugging (config.server.log_level = debug), but did see anything relevant. Also, when i run Promtail with the default scrape configs, i do see k8s namespace in Loki. Isn’t it saying that the k8s namespace are indeed captured?
  2. I tried that. I still get the same result - no logs in Loki.
5

This one worked for me:

---
config:
  positions:
    filename: /run/promtail/positions.yaml
  clients:
    - url: http://loki-gateway/loki/api/v1/push
      tenant_id: 1
  snippets:
    pipelineStages:
    - match:
        pipeline_name: "drop-all"
        selector: '{namespace!="kube-system"}'
        action: drop