Promtail /run/promtail/positions.yaml, permission denied

Hi everyone,

I’m trying to install promtail on a 3 node rke cluster, each node having selinux with Enforce on.

I keep getting error="lstat /run/promtail/positions.yaml: permission denied" which disappear when I disable selinux on every node.
/run/promtail is a hostPath on each node, which seems to be the default values for the grafana/promtail chart.

version:
loki-2.12.2 (chart) v2.5.0 (app)
promtail-6.0.2 (chart) 2.5.0 (app)

1 Like

I’m facing the same problem when upgrading Grafana/OKD Helm in OKD 4.11.x.
I tried configuring in the helm values:

config:
  positions:
    filename: /tmp/promtail/positions.yaml

Which has no effect at all, because the /etc/promtail/promtail.yml config is written from a SecretMap, which is not updated when re-deploying the chart.

You have to change:

configmap:
  enabled: true

first, and then change the setting directly in the ConfigMap.

But this also doesn’t work, because /tmp/promtail/ is read-only.
So I tried to override the volume mounts:

defaultVolumes:
  - name: run
    hostPath:
      path: /tmp/promtail   # Important for config.positions.filename
  - name: containers
    hostPath:
      path: /var/lib/docker/containers
  - name: pods
    hostPath:
      path: /var/log/pods

Which also doesn’t work, because it changes the host volume, not the pod volume.
I stuck with default settings and disabled SELinux on the nodes:

setenforce Permissive

You need to add this to your custom values.yaml:

podSecurityContext:
  runAsUser: null
  runAsGroup: null
  seccompProfile:
    type: RuntimeDefault
  seLinuxOptions:
    type: spc_t