I’m trying to make grafana to work with ingress, but whatever I do, I get an 504 Gateway timed-out error.
I’m using the helm chart from here and modify the values files to setup ingress, from here: https://grafana.github.io/loki/charts/
Here is what happens:
- everything gets deployed fine, no errors whatsoever
- when I curl grafana by pod IP, I get a response
- when I curl grafana by cluster IP, I get a response
- when I curl grafana by ingress IP (external), I get 504 gateway timed-out error
- no errors in grafana pod, kubelet, k8s events or any other place I could think of
Worths mentioning that:
- I tried both using an ingress created by helm, and using a “separate” one (outside helm config); same 504 error
- when I set the service type to LoadBalancer (kubectl edit) , Grafana works
- I tried the ingress config with both “host” and path; each time I get 504 error; please note that when I use the same ingress object to point to another service+port, it works like a charm
Grafana image tag: 7.1.3
Any idea what I might be doing wrong?