We have Alerting configured and working nicely.
Do we have an API to check alerts health to detect if Alerting is down or grafana is down in either case like we test Endpoints with Postman.
How to configure this and do we have any tool integration to test same.
Yes, Grafana provides an API to check the status of alerts and the Grafana instance itself. You can use the /api/alerts endpoint to query alert states and ensure they are functioning. For checking Grafana’s overall health, you can use the /api/health endpoint.
To test the status, you can use tools like Postman to send GET requests to these APIs. For continuous monitoring, you can integrate Grafana with tools like Prometheus, which can track Grafana’s health and alert you when issues arise. You can also use external monitoring services such as UptimeRobot for endpoint health checks.
Hi,
As far as I understand you’d like to have something like “Grafana alert didn’t appear and it should have, so the alerting / Grafana is down” kind of thing? AFAIK Grafana does not have any built-in heartbeats (it would be also weird to monitor if Grafana is up with Grafana). You can of course configure an alert that would send a notification to your checker (e.g. if you’re using OpsGenie, Pager Duty, or something like this - I bet they have heartbeats functionality). I’m using a custom service that increases a metric, by that triggers Grafana alert and then checks if alert appeared in desired place - you could probably do something similar.
I have it set as a cronjob and I can also monitor when was the last time the job was executed. Is it perfect? I don’t think so, but (unless someone comes with a better solution) that’s the best I could think of if you’re not using some kind of OpsGenie / Pager Duty / anything.