I am trying to define a bunch of alerts in Grafana, but I keep seeing this error:
Error = invalid format of evaluation results for the alert definition A: looks like time series data, only reduced data can be alerted on.
The main issue with this is that the labels coming from Prometheus are missing. Only the Grafana defined labels are available. My alert definition looks like this:
- alert: JiraJMXMissing
expr: absent(up{service="jira-jmx"})
for: 5m
labels:
severity: critical
annotations:
summary: Jira JMX Service is missing
description: "Jira JMX Service is missing.\\n LABELS = {{ $labels }}"
Error = invalid format of evaluation results for the alert definition A: looks like time series data, only reduced data can be alerted on.
This error indicates that your alert rule is trying to evaluate a threshold condition against time series data, not a single numeric value. The threshold condition needs to evaluate to one single value per alert instance.
You need to either modify the PromQL query so it returns a single value, or use a Reduce expression in Grafana Alerting to aggregate the time series into a single number.
up{service="jira-jmx"} detects whether the jira-jmx target is up or down. Wondering if you need additional labels in this case. For this case, I recommend reviewing Grafana’s guidance on how to handle missing time series and connectivity issues:
