I have created a kubernetes cluster on my local machine with one master and at the moment zero workers, using kubeadm as bootstrap tool. I am trying to get Prometheus (from the helm packet manager) and Kuberntes matrics together to the Grafana Kubernetes App, but this is not the case. The way I am setting up the monitoring is:
- Open grafana-server at port 3000 and install the kuberntes app.
- Install stable/prometheus from helm and use custom YAML file.
- Adding Prometheus data source to Grafana with IP from kubernetes Prometheus service (or pods, tried both and both works well) and use TLS Client Auth.
- Starting proxy port with kubectl proxy
- Filling in all information needed in the Kubernetes Grafana app and then deploy it. No errors.
- All kubernetes metrics are showing, but not the Prometheus metrics.
If kubernetes proxy connection is stopped, Prometheus metric can be seen. There are no problems connecting to the Prometheus pod IP in the web browser when kubernetes proxy is running. Does someone have any clue what I am doing wrong?
