How to connect Loki (Installed by helm chart Single Binary) with Grafana

After successfully installing Loki using the helm chart (values at the end), the next step is to connect Loki as a Grafana data source. Here it’s become tricky since Loki has a lot of components, but after reading this: Helm Chart Components | Grafana Loki documentation it’s become clear that Gateway service is the one that needs to connect to.

After adding it Grafana returns the error:

Unable to fetch labels from Loki (Failed to call resource), please check the server logs for more details

The Loki installed on namespace “loki”, and this is how I’ve tried to add it: http://loki-gateway.loki.svc.cluster.local

How to debug/troubleshoot? where to start?

The Gateway port is 80, so when having http no need to set a port.On my installation:

kubectl:

Client Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.0", GitCommit:"a866cbe2e5bbaa01cfd5e969aa3e033f3282a8a2", GitTreeState:"clean", BuildDate:"2022-08-23T17:44:59Z", GoVersion:"go1.19", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v4.5.7
Server Version: version.Info{Major:"1", Minor:"25", GitVersion:"v1.25.9", GitCommit:"a1a87a0a2bcd605820920c6b0e618a8ab7d117d4", GitTreeState:"clean", BuildDate:"2023-04-12T12:08:36Z", GoVersion:"go1.19.8", Compiler:"gc", Platform:"linux/amd64"}

helm:

version.BuildInfo{Version:"v3.12.1", GitCommit:"f32a527a060157990e2aa86bf45010dfb3cc8b8d", GitTreeState:"clean", GoVersion:"go1.20.4"}

values.yaml:

loki:
  enabled: true
  global:
    image:
      registry: localregistry.domain.local:8084
  ingress:
    enabled: true
    ingressClassName: "nginx"
    paths:
      write:
        - /api/prom/push
        - /loki/api/v1/push
      read:
        - /api/prom/tail
        - /loki/api/v1/tail
        - /loki/api
        - /api/prom/rules
        - /loki/api/v1/rules
        - /prometheus/api/v1/rules
        - /prometheus/api/v1/alerts
      singleBinary:
        - /api/prom/push
        - /loki/api/v1/push
        - /api/prom/tail
        - /loki/api/v1/tail
        - /loki/api
        - /api/prom/rules
        - /loki/api/v1/rules
        - /prometheus/api/v1/rules
        - /prometheus/api/v1/alerts
    hosts:
      - loki.dc-infra.nano-di.local
  write:
    persistence:
      storageClass: "netapp-storage"
  read:
    persistence:
      storageClass: "netapp-storage"
  backend:
    persistence:
      storageClass: "netapp-storage"
  singleBinary:
    persistence:
      storageClass: "netapp-storage"
  commonConfig:
    replication_factor: 1
  storage:
    type: 'filesystem'

singleBinary:
  replicas: 1

Chart.yaml dependecies:

dependencies:
  - name: loki
    condition: loki.enabled
    repository: https://grafana.github.io/helm-charts
    version: 5.8.3

Hey,
Have you disabled the gateway? Ingress and gateway should not be enabled at the same time.

In addition, if you want to run a single binary you should configure only the filesystem like here:

and for the ingress, I think you can comment out the read and write paths under ingress and use only the singleBinary.
Then your Loki endpoint will look like this:
http://loki.loki.svc.cluster.local

A bit late response but I hope it helps.