How to graph alert events by alert type and key/values in the alert event


tldr# I like to graph alert events and see my alert history by alert type (firing, normal, pending, data, error) and other characteristics that are key/values in the alert event. How do I get my alert history in a Grafana dashboard? tldr#

The Alert visualization dashboard is somewhat limited to showing only current alerts. I tried to look in the documentation on storing and reporting on alert history, but it’s a bit unclear how I should set this up. I have a self-hosted instance of:

  • Gafana Server
  • Prometheus
  • Loki
  • Grafana Agent installed on a couple of nodes.
  • Promtail (in some cases were I do not have a Grafana-Agent on the local machine)

I assume that because I’m self-hosting, I do not get any nice alert statistics on the Alerting Home Screen shown in the Cloud version of Grafana.



I’ve figured it out. Grafana can export the state history to Loki.

enabled = true
backend = "loki"
loki_remote_url = "http://localhost:3100"

enable = alertStateHistoryLokiSecondary, alertStateHistoryLokiPrimary, alertStateHistoryLokiOnly
1 Like