I’ve installed Loki using the Loki-Stack chart from Grafana Community Kubernetes Helm Charts | helm-charts, and it appears that Loki is not responding correctly via it’s API. In grafana when I try to add it as a datasource, the logs show:
logger=context userId=0 orgId=0 uname= t=2024-04-24T20:32:14.537775062Z level=info msg="Request Completed" method=PUT path=/api/datasources/uid/adjo887paz4lcc status=401 remote_addr=192.168.7.59 time_ms=1 duration=1.307594ms size=105 referer=https://grafana.internal.domain.com/connections/datasources/edit/adjo887paz4lcc handler=/api/datasources/uid/:uid status_source=server
logger=tsdb.loki endpoint=checkHealth pluginId=loki dsName=loki dsUID=adjo887paz4lcc uname=admin fromAlert=false t=2024-04-24T20:32:14.702930029Z level=info msg="Prepared request to Loki" duration=57.202µs queriesLength=1 stage=prepareRequest runInParallel=false
logger=tsdb.loki endpoint=checkHealth pluginId=loki dsName=loki dsUID=adjo887paz4lcc uname=admin fromAlert=false t=2024-04-24T20:32:14.706589644Z level=error msg="Error received from Loki" duration=3.510431ms stage=databaseRequest statusCode=400 contentLength=65 start=1970-01-01T00:00:01Z end=1970-01-01T00:00:04Z step=1s query=vector(1)+vector(1) queryType=instant direction=backward maxLines=0 supportingQueryType=none lokiHost=loki.logging.svc.cluster.local:3100 lokiPath=/loki/api/v1/query status=error error="parse error at line 1, col 1: syntax error: unexpected IDENTIFIER" statusSource=downstream
logger=tsdb.loki endpoint=CheckHealth t=2024-04-24T20:32:14.706674084Z level=error msg="Loki health check failed" error="error from loki: parse error at line 1, col 1: syntax error: unexpected IDENTIFIER"
Even if I port-forward the service I get a 404 on just attempting to connect and the same result from trying to curl the same path as grafana.
* Trying 127.0.0.1:3100...
* Connected to 127.0.0.1 (127.0.0.1) port 3100
> GET /loki/api/v1/query HTTP/1.1
> Host: 127.0.0.1:3100
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/1.1 400 Bad Request
< Content-Type: application/json; charset=UTF-8
< Date: Wed, 24 Apr 2024 20:34:29 GMT
< Content-Length: 43
<
* Connection #0 to host 127.0.0.1 left intact
parse error : syntax error: unexpected $end%