Grafana 502 with msg="Request Completed" when sending /query or /query_range to Thanos

shenwan · February 4, 2021, 4:49pm

Hi,

I’m seeing this 502 errors in Grafana log and would like to understand what is going on:

t=2021-02-04T16:32:10+0000 lvl=info msg=“Request Completed” logger=context userId=86 orgId=1 uname=shen.wan method=POST path=/api/datasources/proxy/45/api/v1/query_range status=502 remote_addr=10.40.44.22 time_ms=598 size=0 referer=“https://grafana.myorg.internal/d/yAZ-vPBMk/grafana?orgId=1&refresh=1m”

There are usually ~10 identical log entries (except the time_ms field) like this within a second.

We deploy Grafana to query Thanos in GKE. In Envoy we do not see 502, just 500 and

DC: Downstream connection termination.

Does Grafana terminate the connection and somehow changed 500 to 502? And why it says “Request Completed”? Does it get full or partial result?

wlargou · February 4, 2021, 6:52pm

Hi,

There some related issues on GitHub like this one :

Can you specify grafana version you are using ?

Good Luck

melrose · February 4, 2021, 8:22pm

please update to the latest version of grafana

shenwan · February 5, 2021, 12:42am

Our Grafana version is v7.1.2. Is this a known bug and upgrade would fix it?

wlargou · February 5, 2021, 8:53am

Hi,

As you can see in GitHub, The issue was closed on November 20 and marked as done, so you should go with newer version to test that.

Good Luck

system · February 4, 2022, 4:49pm

This topic was automatically closed after 365 days. New replies are no longer allowed.