Hello,
Could any help me with the following problem? We have Grafana alert based on AWS Cloudwatch Logs query, but sometimes the alert fails with the errors.
The error messages are below:
logger=ngalert.scheduler rule_uid=oEDtfR2Vk org_id=1 version=13 attempt=0 now=2023-02-13T02:09:00Z t=2023-02-13T02:09:12.596886776Z level=error msg="Failed to evaluate rule" error="[plugin.downstreamError] failed to query data: fetching of query results exceeded the max number of attempts" duration=0s
Sometimes all are good, data is returned and the alert works fine.
Does anybody have an idea?
Other useful information:
Grafana version: 9.3.6