Podman containers ran by systemd as user units

Grafana Loki
1

Hi all,

I’m running a couple of (test) containers on a machine. I have these containers run by systemd. As user units specifically. Their unit files are stored in ~/.config/systemd/user/ and enabled and started the usual way.

I have an appropriate promtail config file and run promtail with podman:

podman run --name promtail --volume "$PWD/promtail:/etc/promtail" --volume "/var/log:/var/log" --volume "/etc/machine-id:/etc/machine-id" grafana/promtail:master -config.file=/etc/promtail/config.yaml

The issue is that Loki only picks up on the root level systemd units. All the user units seem ignored. I can confirm that the user unit files work and my containers are running. Am I missing some config param so that promtail picks up on my user units too?

The content of my /var/log/journal:

$ sudo ls -l /var/log/journal/d5650a638b204ff3ba483b83fcbb5c32/
total 221200
-rw-r-----  1 root root 58720256 Oct  8 22:18 system.journal
-rw-r-----  1 root root 75497472 Oct  6 21:35 system@14360f71352d47298d4cdb030edd708f-0000000000000001-0005ea0dac613dc8.journal
-rw-r-----+ 1 root root 16777216 Oct  8 22:18 user-1000.journal
-rw-r-----+ 1 root root 75497472 Oct  4 08:22 user-1000@47b8a2695c784451a5bf2696074e1ad8-0000000000006247-0005ea2306b21d59.journal

My promtail config.yaml:

server:
  http_listen_port: 0
  grpc_listen_port: 0
        
positions:
  filename: /tmp/positions.yaml
        
client:
  url: https://<user:pass>@logs-prod-eu-west-0.grafana.net/api/prom/push
        
scrape_configs:
- job_name: journal
  journal:
    json: false
    max_age: 12h
    path: /var/log/journal
    labels:
      job: systemd-journal
  relabel_configs:
    - source_labels: ['__journal__systemd_unit']
      target_label: 'unit'
    
- job_name: system
  static_configs:
  - targets:
      - localhost
    labels:
      job: varlogs
      __path__: /var/log/*.log

I’m running on rocky Linux 8.6.

Someone have some pointers for me?

1 Like