How is the Loki gateway ingress supposed to work?

Grafana Loki
1

I’d like to make Loki reachable from outside my Kubernetes cluster but it feels impossible. I have configured the following in my Helm chart:

gateway:
  ingress:
    enabled: true
    ingressClassName: traefik
    hosts:
      - host: loki.local
        paths:
          - path: /
            pathType: Prefix

After deployment all pods seem to be fine and I can reach http://loki:3100/ready from a pod within my cluster. However, this does not work:

curl -H “Host: loki.local” http://127.0.0.1/loki/api/v1/ready

<html>
<head><title>502 Bad Gateway</title></head>
<body>
<center><h1>502 Bad Gateway</h1></center>
<hr><center>nginx/1.29.1</center>
</body>

It’s not a 404 so clearly something is happening. I have looked at the helm chart reference but I can’t make sense of this at all.

2

How are you deploying your Loki cluster?

In general, Loki has two major data path, read and write, and they each have their own API paths (see Loki HTTP API | Grafana Loki documentation). From the gateway perspective, you’ll want to send the read APIs to the read path (usually the read svc), and the write APIs to the write path. You can see the nginx configuration in the loki helm chart for a rough idea how it’s done: loki/production/helm/loki/templates/_helpers.tpl at 1e0d525c4ad3f8fd0b6839d6eb8c7ba95117e207 · grafana/loki · GitHub

3

Hi Tony,

I believe I have figured it out, the problem was that the Nginx configuration tries to reach Loki via a cluster internal hostname that includes the namespace specified for the Helm chart. But in my case I did not specify a namespace in the Helm chart but in a kustomize overlay instead. That did then not make its way into the Nginx config. So my question was laking in information and I believe what I have described should work usually.