The query (/api/v1/series) to Prometheus (HTTPS) in Variables doesn't send the Authorization

Grafana Developers & API
,
1

Grafana/9.3.2

I set a query variable in a dashboard to get data from a https Prometheus. But it gets 401 since the query doesn’t send the Authorization to Prometheus.

The datasource works well when Grafana runs /api/v1/query. The access log from Prometheus is like -

10.90.23.174 - monitoringuser [22/Dec/2022:15:05:10 +0000] "POST /api/v1/query?query=chrony_update_interval%7Bhost%3D%22%22%7D%2B0&time=1671721500 HTTP/1.1" 200 87 "-" "Grafana/9.3.2"

But when it comes to Variables, the access log is like -

10.90.23.174- - [22/Dec/2022:15:05:24 +0000] "POST /api/v1/series HTTP/1.1" 401 381 "http://10.90.23.170:30300/d/cntp/cntp?orgId=1&refresh=1m&editview=templating&editIndex=1" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Safari/537.36"

There is no username which means Grafana doesn’t send the Authorization, so it gets a 401 response.

However, for a http Prometheus, it works well calling /api/v1/series with an Authorization.

2

I even can call directly Grafana’s API - http://10.90.23.170:30300/api/datasources/147/resources/api/v1/series?match[]=system_uptime{}&start=1671675209&end=1671696809 and get response. But when I click “Run query” on the Variables page, it failed with 401.

3

Fixed the issue. The root cause is on a reverse proxy in the front of Prometheus which checks the HTTP_REFERER. The difference between a query in Variables and Dashboard is HTTP_REFERER. The HTTP_REFERER in a query of Variables is a URL of Grafana API, not a URL of Prometheus API.