Since an (automatic) upgrade to version 8.0.1 (62e720c06b), my Alerts are no longer working. Actually, the error being thrown is Execution Error: request handler error: failed to query data: request failed, status: 401 Unauthorized
The related datasource is a graphite DB, running in a separate container on the same machine.
To be clear: nothing has changed in the configuration itself: The update to 8.0.1 caused a flurry of Alert emails. with the aforementioned error.
The datasource seems to be correctly setup, as Grafana tells me ‘Datasource is working’ and data indeed is coming in.
It seems to me that Alerts are running a separate query, and that they currently try to access the DB without using the configured username and password.
-
What Grafana version and what operating system are you using?
8.0.1, running inside a docker container (grafana/grafana:latest) on an AWS EC2 instance with Amazon Linux 2. -
What are you trying to achieve?
Getting alerts to work again, as they did before -
How are you trying to achieve it?
So far, I have tried the following: -
restarted both the grafana and graphite containers
-
restart whole server
-
Paused and restarted alerts
-
Recreated alerts
-
What happened?
The errors keep happening. -
Can you copy/paste the configuration(s) that you are having problems with?
Datasource
Alert
-
Did you receive any errors in the Grafana UI or in related logs? If so, please tell us exactly what they were.
The related log entry:
t=2021-08-06T09:03:02+0000 lvl=eror msg="Alert Rule Result Error" logger=alerting.evalContext ruleId=34 name="API Status" error="request handler error: failed to query data: request failed, status: 401 Unauthorized" changing state to=alerting
