Hi
How do I tell if the alert on Grafana Cloud is triggering due to execution error or timeout?
When there is no data, the alert state is set to OK. According to the alert history, the alert state does not transition to pending when going from Normal to Alerting. Looking at the data during the period that the alert was triggering, the data is outside the alert range. There are no labels on the alert in question, when there are if state does transition through Pending to Alerting. Therefore I suspect that the alert is being triggered by execution error or timeout, but I can’t see a way to identify this explicitly.
The alert is node_filesystem_avail_bytes{fstype!~"tmpfs|fuse.lxcfs"} < 1024 * 1024 * 200
, which is then reduced using the function Last in strict mode. Therefore I don’t know why this would have an execution error or timeout.
Thank you