Promtail unable to capture logs in Openshift 4.6

Hi, I’m not sure anyone had been successfully setup PLG stack in openshift, it suppose to be similar to typical k8s setup with given required cluster roles. However, I cannot found any useful information when working on this setup. The debug logs just shown starting provider and that’s it, no label captured. I’m not sure it is compatibility problem or just a recent bug.

    server:
      http_listen_port: 9080
      grpc_listen_port: 0

    positions:
      filename: /var/log/promtail-positions.yaml

    clients:
      - url: http://loki.caas.svc.cluster.local:3100/loki/api/v1/push

    scrape_configs:
    - job_name: kubernetes-pods-name
      kubernetes_sd_configs:
      - role: pod

nothing show in debug logs.

level=info ts=2020-12-27T13:10:11.354461914Z caller=kubernetes.go:263 component=discovery discovery=kubernetes msg="Using pod service account via in-cluster config"
level=debug ts=2020-12-27T13:10:11.355738697Z caller=manager.go:195 component=discovery msg="Starting provider" provider=kubernetes/0 subs=[kubernetes-pods-name]
level=info ts=2020-12-27T13:10:11.355970873Z caller=server.go:225 http=[::]:9080 grpc=[::]:38047 msg="server listening on addresses"
level=info ts=2020-12-27T13:10:11.356097981Z caller=main.go:110 msg="Starting Promtail" version="(version=2.1.0, branch=HEAD, revision=1b79df375)"

If i change the role to node, at least i got something to work with.

level=info ts=2020-12-27T13:08:20.003733862Z caller=filetargetmanager.go:241 msg="no path for target" labels="{__address__=\"192.168.82.9:10250\", __meta_kubernetes_node_address_Hostname=\"testingocpn02-cp.ocpcluster.local\", __meta_kubernetes_node_address_InternalIP=\"192.168.82.9\", __meta_kubernetes_node_annotation_csi_volume_kubernetes_io_nodeid=\"{\\\"cephfs.csi.ceph.com\\\":\\\"testingocpn02-cp.ocpcluster.local\\\"}\", __meta_kubernetes_node_annotation_k8s_ovn_org_l3_gateway_config=\"{\\\"default\\\":{\\\"mode\\\":\\\"shared\\\",\\\"interface-id\\\":\\\"br-ex_testingocpn02-cp.ocpcluster.local\\\",\\\"mac-address\\\":\\\"00:50:56:9b:07:2c\\\",\\\"ip-addresses\\\":[\\\"192.168.82.9/23\\\"],\\\"ip-address\\\":\\\"192.168.82.9/23\\\",\\\"next-hops\\\":[\\\"192.168.82.1\\\"],\\\"next-hop\\\":\\\"192.168.82.1\\\",\\\"node-port-enable\\\":\\\"true\\\",\\\"vlan-id\\\":\\\"0\\\"}}\", __meta_kubernetes_node_annotation_k8s_ovn_org_node_chassis_id=\"7f6c17de-da5a-450f-8bc6-c96b74320a4c\", __meta_kubernetes_node_annotatio...

Did you figure out how to solve this problem? I am running into the same issue @levindecaro

I use fluent-bit with grafana loki plugin at now.

try looking into the answers here, might help you