Hello Everyone,
I am new to Grafana tempo(2.5.0), I have installed Grafana tempo distributed on k8 using helm charts.
I have enabled s3 storage for storing the traces and they are getting stored and I can see them.
I have also, added Tempo as a datasource in Grafana- connection test is successful.
But when I try to see any traces in Grafana dashboard- it shows nothing.
Below are the logs of ingestor/query ::
ingestor logs-
level=warn ts=2024-09-02T11:41:39.707274717Z caller=instance_search.go:71 msg=“block does not support search” blockID=747e6d88-ed7a-436f-8396-safkhjasdf
frontend-query logs-
caller=handler.go:133 tenant=single-tenant method=GET traceID=33c345441618fc28 url=“/api/search?q=%7Bduration%3E1%7D&limit=20&spss=3&start=1725255698&end=1725277298” duration=11.453819ms response_size=0 status=500
level=warn ts=2024-09-02T11:41:39.714334342Z caller=server.go:2136 trace_id=33c345441618fc28 msg="GET /api/search?q=%7Bduration%3E1%7D&limit=20&spss=3&start=1725255698&end=1725277298 (500) 11.750627ms **Response: "unsupported**\n" ws: false; Accept: application/json, text/plain, /; Accept-Encoding: gzip, deflate, br, zstd; Accept-Language: en-US,en;q=0.8; Priority: u=1, i; Sec-Ch-Ua: "Chromium";v="128", "Not;A=Brand";v="24", "Brave";v="128"; Sec-Ch-Ua-Mobile: ?0; Sec-Ch-Ua-Platform: "Windows"; Sec-Fetch-Dest: empty; Sec-Fetch-Mode: cors; Sec-Fetch-Site: same-origin; Sec-Gpc: 1; User-Agent: Grafana/11.2.0; X-Forwarded-For: 192.168.0.120, 192.168.131.18, 192.168.131.18; X-Forwarded-Server: ingress-traefikv2-b7558865c-sh5xq; X-Grafana-Org-Id: 1; X-Grafana-Referer: https://monitoring.dev.net/d/c4ef519b-a77e-48f9-9b0f-7f83f8d6f398/api-traffic?orgId=1; X-Real-Ip: 192.168.0.120;
Query logs
caller=server.go:2136 trace_id=33c345441618fc28 msg="GET /querier/api/search?blockID=9ad30469-fbf3-4e5e-83a8&dataEncoding=v2&encoding=zstd&end=1725277298&footerSize=0&indexPageSize=256000&limit=20&pagesToSearch=5872&q=%7Bduration%3E1%7D&size=17856&spss=3&start=1725255698&startPage=0&totalRecords=1&version=v2 (500) 758.642µs Response: "unsupported\n" ws: false;
I am not sure, what’s going wrong. If someone has ever came across this problem in past. Can guide me here?
If you need more details, please let me know.
Thanks in advance
Vijay
