Grafana using the new /ds/ API and not working

Hi, I’ve recently updated from:
grafana-8.2.2
to:
grafana-8.4.2

But it seems all data sources are rendered invalid and I get the following error inside Grafana:
Error reading Prometheus: bad_response: readObjectStart: expect { or n, but found <, error found in #1 byte of …|<!DOCTYPE h|..., bigger context ...| <meta content="text/html;c|…

As far as I could tell the difference comes from where Grafana makes the API call for the datasource. In 8.2 it calls:
/grafana/api/datasources/proxy/7/api/v1/query
Now it calls:
/grafana/api/ds/query

Is this a configurable parameter to switch back to the old API? I couldn’t find anything in the docs unfortunately.

1 Like

After spending a couple of hours debugging it seems the issue is caused by the 8.3.0 UPDATE, one of the request headers which we use for authentication in our system is no longer being proxied to the datasource.

Haven’t found a fix for it yet but atleast I’m steering into the right direction now, will update the thread if I find a fix in the meantime.

1 Like

Seconding this issue, as we’re seeing that the annotations api calls does not support the direct browser based data sources after upgrading to 8.3.6. This appears to be a breaking change that we weren’t expecting to have to deal with until Grafana 9.x was introduced